Main Site News at A.P.Lawrence.com

How to upgrade Kerio Connect Mail Server

Connect,Kerio,Mail 2013/06/18

Most of the time, upgrading Kerio Connect involves nothing more than downloading the installer for the new version and running it. The process is quick and painless - a very brief interruption of services and then you are back in business.

Of course you should always have current backups before any upgrade, but there is an extremely low likelihood of any trouble whatsoever. Follow the general directions at How do I update Kerio Connect when a new version is available?

There are circumstances where a bit more planning and work may need to be done. One is when you are moving to new hardware or changing operating systems. Another is when you have delayed doing upgrades for some time. We'll look at both of those and more.

New hardware or operating system change

Most of that is covered in Kerio's KB article How do I move Kerio Connect from one machine to another (or change Operating Systems)?, but there are a few things that doesn't mention.

One is what to do when you plan a version upgrade at the same time as the hardware change. I suggest doing the version upgrade AFTER the physical move. You can download any older version of Connect; see How do I get older versions of Kerio software?

The other issue is how to get the Store data to the new machine. See my Unsupported use of KMSRECOVER for moving Kerio Connect and Transferring a Kerio Connect Server with rsync for more details on that.

Delayed upgrades

If you are behind by several versions, that's usually no different than a simple upgrade: just download the latest version and install it. You'll want to read the appropriate parts of http://www.kerio.com/connect/history and http://www.kerio.com/connect/history/older to get a feel for things that have changed. After any upgrade you will want to run through your configuration to become familiar with any changes there and investigate any new features you'll want to configure. I can help you with that if you are one of my customers - as with any support issues, there is no charge for that.

If you are really far behind - older than version 6.7.3 Patch 1 (January 2010), you'll need to do your upgrade in steps - see the How do I update Kerio Connect when a new version is available? article.

Client upgrades

If your users are not using the Kerio Outlook Connector, there is nothing to be done on the client side.

If you are running a recent version of Connect, the Outlook Connector will attempt to upgrade itself automatically. I wish I could say that it always succeeds, but unfortunately it doesn't.

One reason might be incompatibility - your Outlook might be too old or too new. See the End User tab at http://www.kerio.com/connect/requirements for current requirements.

Your users can cause problems too. The Kerio manuals say this:

If a new version is available or the versions are different on the server and the client, a recommendation for update of Kerio Outlook Connector appears. Once this dialog is confirmed, update is performed, followed by an automatic restart of MSOutlook. The whole update including restart should take up to two minutes, depending on connection data-flow speed.

Did you notice the "Once this dialog is confirmed" part? Users sometimes ignore that upgrade message completely. And, of course, even if they do not, things can go wrong and the client update can fail.

If you are only going from the not-quite-latest version to the latest, that might not matter: the old software may work with the new server. It may not, though, or it may work but cause performance issues or subtle bugs.

So, what do you do if a client update fails?

Probably the first thing to try is a manual download and manual install. Use the MSI version rather than the .exe:

You should also kill off any running installs and check Task Manager to see if any msiexec.exe processes are running. Kill them if they are.

If that still fails, try removing both the Kerio Updater software and the Connector with Control Panel and then try a fresh install. Under some conditions, Kerio support may want you to use a special "ktuninstaller.exe" that they can provide.

Note: the MSI installer comes in 32 bit or 64 bit versions. The 32 bit can work with any version of Outlook, but you can't use the 32 bit with 64 bit Outlook.

There is also a separate MSI package available for just the Kerio Updater Service (this is the piece that runs as a service and is supposed to do the Connector upgrades when needed). You can try that, too.

Still can't get it working? Call me or Kerio, check the forums and research the KB articles. If the MSI package is failing, a little extra logging might help you or us figure out why:

msiexec /i kerio-connect-koff-(....)win32.msi> /lv C:\temp\kerio_install.log

See Command-Line Switches for the Microsoft Windows Installer Tool also.

There are more logs under %programdata%/Kerio - Kerio Support may want those also.

The Outlook Cache

Several connector upgrades have required rebuilding the local cache. That can be a very time consuming procedure - the general recommendation has been to use webmail while it rebuilds.

There is another way, however. If you remove the cache before the upgrade, you can start using Outlook again immediately. Doing that requires a bit of work though. The first thing is to find your Outlook cache files. KERIO OUTLOOK CONNECTOR CACHE FILES ARE IN A DIFFERENT LOCATION THAN OUTLOOK CACHE FILES.

Under C:\Documents and Settings\YourLogin\Local Settings\Application Data\Kerio\Outlook Connectorwill be a ditectory for each of your profiles. You may need to turn on "Show Hidden Files and Folders" to navigate to there.

Once there, it's the *.FDB files that you'd remove and you may need to kill off the KoffBackend.exe process to do that.

With those removed, Outlook will need to resynch, but you at least can use it right away:

I hope this helps and of course I am always available for assistance.

Comments: Click Here.

Want to showcase your product to our audience? Check our advertising options.

Many of the products and books I review are things I purchased for my own use. Some were given to me specifically for the purpose of reviewing them.

I resell or can earn commissions from the sale of some of these items. Links within these pages may be affiliate links that pay me for referring you to them. That's mostly insignificant amounts of money; whenever it is not I have made my relationship plain. If you have any question, please do feel free to contact me.

Samepage - Redefining how people create and share information

Kerio Mail Server, Firewall and more

E-Mail to everyone in your Kerio Connect domain

Connect,Kerio,Mail 2013/06/17

Sending an email that goes to everyone in your Kerio Connect domain sounds simple, doesn't it? It actually is - you just need a group that includes everyone. That's easy enough, especially if you use a directory service that creates that group automatically. In that case, you just import the group from the directory service and assign it an email address.

If you don't use a directory service, you'll need to remember to update that group when you add or delete email users, but as you can select everyone at once, that's no great hardship.

Another way to accomplish the same thing is to create an "everyone" alias that delivers email to a public folder rather than an email address. Share that folder with whomever you like.

That also has the advantage of fine tuning who sees and controls these emails

There is a problem, though: if you have an "everyone" group or alias, people from the outside world can spam all your users at once. If that's not acceptable, what can you do?

There are actually a number of ways to do this. Each has its own advantages and disadvantages, though.

A shared Contact group/distribution list

A distribution list (now called "contact group") is an internal group. See Kerio Connect Webmail Distribution lists

Unlike a group, these are not available from outside.

These are created by individual users, but can be put into Public Folders and thereby shared with everyone. See "Consider using Kerio Connect Contact Groups instead of Mail Groups"

A mailing list

Kerio Connect's mailing lists are an under-utilized feature. As you can control who can post to the list and the people who receive it, this can be a great way to have an "everyone" address. For example, you could restrict posting (sending email to the group) to an administrative group only.

On the members side (people who receive the email sent to the list), maintenance is a bit annoying, but a csv import function makes that a little easier and the list is just a text file on the server, so you could write an external script to keep it updated.

A domain limited user

This method has the disadvantage of consuming a user license (none of the others do), but does have some interesting aspects.

You create an "everyone" group as before, but you only put one user in it. This is a new user who forwards everything to whomever you want. The "everyone" group is set to be limited to your domain

Note that the user you put in the group is now prevented from sending or receiving outside of your domain, so no outside person can send email to him directly either, therefore don't use an existing user!

You'll need to update this user's forwarding as you add and delete users, of course.

A post at the Kerio forums suggests obfuscating this by having that user forward to an oddly named group that includes everyone. It could also forward to oddly named alias that delivers to a public folder.

Comments: Click Here.

Want to showcase your product to our audience? Check our advertising options.

Many of the products and books I review are things I purchased for my own use. Some were given to me specifically for the purpose of reviewing them.

Samepage - Redefining how people create and share information

Kerio Mail Server, Firewall and more

Hosted vs. On Premise Email

Connect,Kerio,Mail 2013/06/14

Barely a week goes by where someone doesn't mention either Office 365 or Google Apps to me (or both). You might think that I'm adamantly opposed to hosted email because I sell on-premise solutions, but I'm not. It can make sense for some people and there are even some advantages, but just as having an on-premise mail server has its good points and its bad, so does out-sourcing your email to someone else.

By the way, I do need to mention that you can self-host. That is, if the idea of email in the cloud appeals to you because availability or for whatever reason, there's no reason that you can't put up a server at Amazon or whatever and retain control of your email. See my Hosted services vs. self-hosting article for more on that.

Here, I want to look at the things you should consider before shutting down your in-house mailserver.

Disclaimer: I've been selling and supporting mailservers for more than twenty years. Of course I have a bias toward the things I sell. I always try to see that in myself so that I can keep it out of articles like this, but you should be aware of my prejudices.

Cost

Although both Microsoft and Google are charging far less than many other hosting companies, there's absolutely no contest against on-premise unless you only have a very small number of users. I did some cost comparisons at my Hosted Google Apps Gmail vs. in-house Mail Server post a few years ago. Hardware costs have of course decreased since then and while support costs may have increased, the need for operating system support has actually lessened and can even almost disappear entirely with packaged virtual machines.

The cost for a 50 user hosted email will be approximately $2,500.00 a year. That's a fair pile of money and that is that. Contrast that with the typical maintenance cost for a Kerio mailserver of the same size, which would less than $600.00. Yes, that's not counting hardware or support, but even if you go overboard on estimating that, you'll always be less money.

The cost gap becomes even wider as your user base goes up. A 100 user Kerio system renews at around $1,100.00 yearly vs. $5,000.00 or more for hosted email. Hardware and support costs aren't any different.

That doesn't count extras like more storage space, archiving, virus scanning..

Support

"Wait just a minute", you are probably saying, "A 100 user base obviously requires more support than a 50 user base!"

Well, sure. But those are support costs you have whether your email is in-house or out. Users who can't remember passwords, users who misconfigure clients - those are still your problems. So are the users who mangle email addresses they type, set their reply-to incorrectly or give out incorrect email addresses to people who want to send you email. Almost everything on the client side (which is where the lion's share of the support is anyway) is still your problem.

And, lucky you, you get some new issues to deal with..

Blacklists

You may have had the unpleasant experience of having your IP blacklisted. That sometimes happens because some spammer was using that IP before you got it, or it could be because you caught a PC virus or were too aggressive with your marketing email. Whatever the cause, it was probably pretty quick to fix and (unless you kept on being sloppy) it may never have recurred.

When you let somebody like Google or Microsoft handle your mail, you are usually sharing an IP with other people. When they misbehave, the resulting blacklisting affects them AND you.

It happens often enough that Google even has an article about it

Quote from http://support.google.com/a/bin/answer.py?hl=en&answer=27642

On rare occasions, you might send mail from your Gmail account, then receive a notice that your mail has bounced because your IP addresshas been blacklisted by the recipient.

I can tell you that it's not all that rare..

More important is that unfortunately, not everybody *bounces*. I don't bounce people on blacklists, I just silently ignore them and I know that other folks do the same. That means that sometimes you won't KNOW that your email never got delivered.

Why is that any different than on your own server? Let's look at two scenarios to see why.

In the first, it's your server that's been blacklisted. As it almost certainly will remain blacklisted until you do something about it, you will find out - some piece of mail will bounce sooner or later. When it does, you fix the problem and , as noted before, unless you make a habit of being spammish, it may not happen again for a long time, if ever.

In the other case, the IP you share with who knows how many other people gets blacklisted because one of them is a spammer. Let's say that's at Microsoft and that they are really on top of it and find out very quickly. Within an hour they identify the bad guy, boot him off and have the blacklist removed.

Everything you send in that period might not get through. After that, everything's fine. During that hour, if nothing actually bounced, you could have emails that were just silently ignored by the recipients. Isn't that fun? Mysteriously missing emails.

Here's the best part: it can happen again next week, next month or even tomorrow! It could happen over and over again and you might never even know..

Compliance

If you are subject to Sarbanes Oxley, HIPPA or other regulations, these are all still your responsibility, too. Need to archive? That's extra cost with most hosting - it isn't when you do it yourself.

Customization

If you are a regular reader of this site, you know that I'm always writing scripts to do this odd thing or the other. That's an ability you usually lose completely when someone else hosts your email.

Security

When your email is in-house, in-house email never leaves your network. Yes, your local administrator can read your email. So can the admin(s) at the hosted site - do you know who they are?

Changed your mind?

It's usually easy to transfer your on-premise email store to someone else. Kerio Connect stores every bit of email and every contact and calendar entry as plain text files and so do most in-house systems (with the notable exception of Exchange). You can almost always get that data to wherever it needs to go.

Will it be just as easy to bring it back? Maybe, maybe not..

Internet Down

Oops - no email. You can still do internal email if you are on premise, but not if you are hosted.

What about people sending email to you? If your on-premise server is unavailable because your internet is dead, your customers know that. They aren't going to be fuming as the hours tick by and that Very Important email they sent doesn't get answered. It got delivered to your hosting provider just fine, but you can't get it - too bad your customers don't know that!

Maintenance Windows

You may not have any control over maintenance windows with hosted email. When they decide something needs to be rebooted, migrated, upgraded or whatever, they usually do what THEY need to do, not what's convenient for you.

System down

Hey, it can happen to anyone - your in-house system or a hosted system. If your data store isn't damaged or you just want email back up and running with or without the existing store, your recovery time with a system like Kerio is just a few minutes longer than the time it takes to find a machine to run it on. If you need that to happen NOW, you can probably do it.

With the hosted provider.. heck, who knows? Yeah, they have an SLA with you and yeah, they want it back up running fast, but it's THEIR priorities that matter, not yours. So if their 5,000 seat customer machine crashes at the same time as one with 1,000 seats, I'd expect they'd replace the big one first. Too bad you are one of the 1,000, isn't it?

So, that's it. Some things to think about. I should also mention that if you insist upon going this way, I can offer hosted Kerio Connect to you also. I can't beat Office 365 prices, but I can come close and I can beat the heck out of them on support, so if that matters to you, give me a call or shoot me an email.

Comments: Click Here.

Want to showcase your product to our audience? Check our advertising options.

Many of the products and books I review are things I purchased for my own use. Some were given to me specifically for the purpose of reviewing them.

Samepage - Redefining how people create and share information

Kerio Mail Server, Firewall and more

X-Envelope-To:; vs. To:;

Connect,Kerio,Mail 2013/06/13

If you've ever poked around in the "Advanced Options" section of your Kerio Connect mail server configuration, you might have wondered about the check-box for "Insert X-Envelope-To header to locally delivered messages".

The newest manuals don't even mention that, but the older versions do have at least this much to say:

Defines if the X-Envelope-To entry will be inserted into the header of messages delivered locally. X-Envelope-To is the original recipient address based on the SMTP envelope. This option is useful especially if there is adomain mailbox in Kerio Connect.

Perhaps a little more explanation is in order.

If you've ever done a "View source", you are certainly familiar with the "To:" header. Normally that contains your email address, but you might sometimes see something very odd like this example:

Mime-Version: 1.0
To:VendorSupport@notyourdomain.com
From: Vendor Support <VendorSupport@notyourdomain.com>
Return-Path:VendorSupport@notyourdomain.com

You are not "VendorSupport@notyourdomain.com", yet the email arrived in your mailbox. How?

The answer is simple enough: "To:" doesn't really matter. If we use the analogy of a physical letter, that "To:" is what's written on the letter itself and is not necessarily what's written on the envelope that told the Post Office how to get that letter to you. Just like a physical letter, email has an outer envelope too.

Most email clients don't let you play any games with what's inside and outside. When you send an email, the envelope address and the "To:" address are usually going to be the same. But they don't have to be.

A little Perl script shows that more plainly:

#!/usr/bin/perl

sendit($ARGV[0], $ARGV[1] );

sub sendit {
my $xenvelope_who=shift;
my $apparently_to=shift;

open(SENT,"|/opt/kerio/mailserver/sendmail $xenvelope_who");
print SENT <<EOF;
Content-Type: text/html
Content-Transfer-Encoding: 8BIT
From: tony\@aplawrence.com
To: $apparently_to
Subject: X-env test

This was just a test

EOF
close SENT;
}

We can use that to munge up "To:"

tmail.pl tony@aplawrence.com nobody@all

Kerio Connect "sendmail" helpfully added the true send-to address anyway, but as you saw in the first "VendorSupport" example I showed, that isn't always going to be the case. That's when we'd want to have "Insert X-Envelope-To header to locally delivered messages" turned on:

By the way, although we used something that at least vaguely looks like an email address, we don't have to:

tmail.pl tony@aplawrence.com not_even_an_email_address_at_all

If you are wondering what "This option is useful especially if there is a domain mailbox in Kerio Connect" is referring to in the old manuals, I suspect it has to do with POP3 downloads, but I have no way to test that right now.

Comments: Click Here.

Want to showcase your product to our audience? Check our advertising options.

Many of the products and books I review are things I purchased for my own use. Some were given to me specifically for the purpose of reviewing them.

Samepage - Redefining how people create and share information

Kerio Mail Server, Firewall and more

Kerio Control Connection Limit Reached Alert

Control,Kerio,Security 2013/06/12

In Kerio Control Configuration ->Traffic Policy -> Security Settings -> Miscellaneous, you'll find a setting for "Connection Limit". That puts a limit on connections and by default it is set to 600. That means that if a machine in your network tries to have more than 600 connections to machines on the Internet, it gets blocked.

Note that this is connections in either direction: outgoing or incoming.

That's a pretty generous limit, as any machine with that many connections may be up to no good. If the connections are legitimate, you are probably already well aware of exactly what those connections are and why that machine needs to make them. In that case, you either need to raise the limit or put a public interface on that machine that doesn't go through the firewall (it's not currently possible to have any exceptions for this limit or to define individual limits for certain machines).

What if there isn't any reason that you know of for this machine to be so active? You suddenly get this and have no idea why:

As noted, this could be bad news. It could indicate a virus on the misbehaving computer, or it could mean that someone on the outside is trying to cripple you by deliberately making useless connections.

Or, it might just be something you forgot about.

In the two most recent cases where I've seen this, that was the case. In both cases, the customers were using Microsoft Domain Controllers and some or all machines were looking to there for DNS. That machine in turn was going out to the Internet to resolve the requests and although neither customer has anything even approaching 600 users, connections last long enough and come quickly enough that now and then the connection limit was reached.

Aside from the alert, this also means that some DNS request failed, though apparently it didn't happen often enough that anyone complained. Probably the affected user was momentarily puzzled and when they tried again, enough older connections had timed out that it went through, so they shrugged their shoulders and got on with their work (or whatever they were doing instead of work).

The solution is simple enough: tell the Domain Controller (or the individual machines) to look up DNS at the Kerio Control box. That isn't an outgoing or incoming connection, so the connection limit counter isn't affected.

Comments: Click Here.

Want to showcase your product to our audience? Check our advertising options.

Many of the products and books I review are things I purchased for my own use. Some were given to me specifically for the purpose of reviewing them.

Samepage - Redefining how people create and share information

Kerio Mail Server, Firewall and more

Multiple Spam folders with Kerio Connect

Connect,Kerio,Mail,Malware 2013/06/11

If you are a Kerio Connect user or administrator, you have certainly seen messages marked as "**SPAM**". You may also be aware that you can change that to prefix spam messages with something else.

For example, here I've changed mine to read "**SPAM** [%s]":

If you are a new to Kerio Connect, you may be puzzled by this. The current manuals don't mention anything about that. Why would I do that?

Actually, it's something that used to be mentioned in the manuals, and fortunately it still works.

What does it do? It makes your Spam messages look like this:

That gives you an immediate visual clue as to the actual Spam rating. If I'm looking for false positives (things marked as Spam that might not be), I'd look at those with "**SPAM** [****]" (4 stars) first, because those have the lowest Spam rating.

It would be nice if you could sort by Subject so that the lower scoring messages came first, but neither Outlook nor Webmail pay any attention to that and instead sort by alphanumerics only.

There is another way, though.

Sort messages into multiple Junk Folders

As you'll see shortly, this isn't something most of us would want to do because it affects out of office settings. Think about all of this before rushing to implement it!

This comes from the tip in the older manuals that suggests that you could write rules to sort messages out into multiple folders based on the Subject line. That involves a bit more work, but we can do it.

You'll need to add Mail filter rules for each condition you want to sort specially. I moved "[****]" and "[*****]" messages to their own folders with these three rules:

* Be sure to use "contains" and not "matches" to avoid interpreting the "*"'s as wildcards.

To allow these new rules to work, you need to disable the default spam rule:

With this all in place, I now have lower scoring spam separate from high scoring spam. That could make it easier for me to quickly check for false positives.

Note: the new rules here are not the same as the default rule. That rule uses a different test and also takes the whitelist into account. You could edit the filters manually to do the same; see Using Sieve for spam in Kerio Mailserver. If you use the personal whitelist, you'd definitely want to do that (and you'd need to update your rules whenever that whitelist changes!)

Look at the "filter.siv" file in your mail directory to see these default rules.

#!1 __WebmailJunk__
if allof (spamtest, not address :all :is "From" "") {
  fileinto "Junk E-mail";
  stop;
}

#!0 __WebmailAutoreply__
#if true {
#  vacation :days 7 "I am out of office today.  Please call 774 213-1199 if urgent\r\n\r\n";
#}

Also, this affects the out of office rule. With the default spam rule, out of office messages won't be sent in reply to messages that are marked "**SPAM**". Unless you move that rule below your new rules, out of office messages will be sent. This requires manual editing of "filter.siv". As the normal out of office message will get moved right back to the top every time it is enabled, you'd probably want to define your own rule, instead.

Another way to handle that is to re-enable the default rule before you enable out of office messages. The disadvantage of that is that your Spam won;'t be sorted out into your new folders, because your rules won't ever be seen. No matter what, this isn't going to be ideal.

Comments: Click Here.

Want to showcase your product to our audience? Check our advertising options.

Many of the products and books I review are things I purchased for my own use. Some were given to me specifically for the purpose of reviewing them.

Samepage - Redefining how people create and share information

Kerio Mail Server, Firewall and more

Whatever happened to Winmail.dat issues?

Connect,Kerio,Mail 2013/06/10

Assuming you are a non-Microsoft user - and why WOULD a smart person like you be a Microsoft person? No, seriously, assuming that you are either not a part of the Evil Empire - um, not a Microsoft user, or at least not always a Microsoft user (maybe you have a tablet or a Mac as a second machine), when was the last time you saw a winmail.dat attachment? It's been a long time since I've seen one or had anyone ask me about that.

Google "winmail.dat" if you are a Microsoft user with no non-Microsoft folks in your addressbook and therefore have no idea what I'm talking about.

It seems strange - that used to come up frequently. People would think the file was an attachment they were supposed to open. There actually could be an attachment embedded in there, apparently, but most of the time it's just Outlook formatting junk and of no use to anyone except another Outlook user.

Those Outlook users can control their outgoing format:

I think maybe Outlook now defaults to HTML rather than RTF (setting it to RTF was what caused those stupid Winmail.dat things). That could explain why I haven't seen any recently. Or if not, a lot of people have changed their settings, which seems less likely.

Some servers like Kerio Connect have a setting to silently decode TNEF (Transport Neutral Encapsulation Format, which is how these things are encoded), so I'd only see this when someone sends something to my Gmail account rather than my normal address. But, I haven't seen one of these in Gmail for some time either: I did a search and the last one I saw was in March of 2009. I forwarded it to myself again to see if maybe Google had started decoding these too, but no, it came through.

The last time somebody asked me a question about this involved Apple mail. They use Kerio Connect, so the decoding wasn't an issue, but (according to what Kerio support determined), the Apple Mail itself was at fault:

The problem is on sender's side - he sends the email encoded in proprietary TNEF format from Microsoft Exchange (winmail.dat). Kerio Connect decodes such messages so it can be displayed on all email clients. The message has two identical parts, one in plaintext, one in RTF. Some clients choose to display RTF event if they are not able to do it correctly. Apple Mail and IOS Mail does this. It is already reported to Apple.

I don't know if current Apple Mail has that issue, but sometime later I came across this:

defaults write com.apple.mail PreferPlainText -bool TRUE

That's supposed to tell Apple mail that you want to see plain text emails. I've never tried it, but am also told that if you have this set and DO want to view (for example) an HTML formatted email as the sender intended you to see it, you can use View -> Message -> Next Alternative. I've never tried that either (I'm not a fan of Apple Mail, sorry).

Kerio Connect users only: The Debug log can turn on extended debugging with the Message decoding option. I turned that on and forwarded myself that RTF message that someone had sent me in 2009. The log noted this:

[09/Jun/2013 15:36:21][15379] {msgdecode} TNEF part of message
/opt/kerio/mailserver/store/queue/00/51b4d933-0000013a.eml decoded.

I supposed it might be more detailed had it encountered some problem, so I decided to try that. I downloaded a winmail.dat file and sent it to myself as an attachment. I confirmed that I got the "decoded" result in the Debug log and then tried corrupting the attachment with various permutations of:

echo "abc" | dd of=t seek=1024 count=3 bs=1 conv=notrunc

(See dd - data dumper if you don't recognize what that is doing)

That DID corrupt the attachment, but all that happened was that Connect left it as an attachment and no message was written to the log. Other logs can report TNEF errors, apparently: I saw this in a forum post about a TNEF issue:

[22/Apr/2009 11:29:58] mail_queue.cpp:
TNEF decoding of message C:\Program
Files\Kerio\MailServer\store/queue/34/49ef4604-0000542f.eml failed:
TNEF stream contains unsupported properties

I suppose that the non-reporting of obvious corruption might bother me more if it had not been so long since I have seen any TNEF, never mind something corrupted!

How about you? Do you still see these critters?

Comments: Click Here.

Want to showcase your product to our audience? Check our advertising options.

Many of the products and books I review are things I purchased for my own use. Some were given to me specifically for the purpose of reviewing them.

Samepage - Redefining how people create and share information

Kerio Mail Server, Firewall and more

Creating Contact Groups from Mail Groups

Connect,Kerio,Mail,Perl,Scripting 2013/06/09

At "Consider using Kerio Connect Contact Groups instead of Mail Groups", I talked about the advantages of Contact Groups for internal lists. If you only had one or two such lists with a small number of contacts in each, you could manually convert them to Contact Groups without much work. But what if you had several dozen to convert? That would be tedious, wouldn't it?

Fortunately, it's not particularly difficult to script this. We first need to understand how Kerio creates Contact Groups, then we need to gather the members of the Mail Groups we want to convert and then (with a little bit more work) we can create new Contact Groups.

(This article is a bit more geekish than usual and the code is longer. It may be of little interest to the average user or administrator.)

Contact Groups Format

Here is an example of what a Contact Group actually looks like on your disk. It's just text; you can examine these things with any text editor.

This one is more complex than we'll actually need because it contains other Contact Groups as well as both individuals and other groups.

Subject: Nested Contact Group
Date: Fri, 7 Jun 2013 08:20:40 -0400
Content-Type: text/x-vcard-dl; charset="utf-8"
Content-Transfer-Encoding: 8bit

BEGIN:VCARD
VERSION:3.0
PRODID:-//kerio.com/Contacts//NONSGML v1.0//EN
X-DL:TRUE
FN:Nested Contact Group
UID:6596500c-5a9d-4788-8aa4-3f7e2ee324c3
BEGIN:X-DL-ITEM
TYPE:REFERENCE
NAME:Robert Lawrence
EMAIL:robert@aplawrence.com
DATA:fd3ab4b6-4fa0-460a-a14c-2eb13651e369/00000003
END:X-DL-ITEM
BEGIN:X-DL-ITEM
TYPE:REFERENCE
EMAIL:nosuchperson@nowhere.com
END:X-DL-ITEM
BEGIN:X-DL-ITEM
TYPE:REFERENCE
NAME:AAGroup
EMAIL:aagroup@aplawrence.com
DATA:fd3ab4b6-4fa0-460a-a14c-2eb13651e369/00000011
END:X-DL-ITEM
BEGIN:X-DL-ITEM
TYPE:LISTREFERENCE
NAME:Contact Group One
DATA:e14df63c-e49e-45dc-ae98-976e6ff66fca/00000198
END:X-DL-ITEM
END:VCARD

Let's break this down so we understand it. After the stuff that identifies the Contact Group itself, we have the first member:

BEGIN:X-DL-ITEM
TYPE:REFERENCE
NAME:Robert Lawrence
EMAIL:robert@aplawrence.com
DATA:fd3ab4b6-4fa0-460a-a14c-2eb13651e369/00000003
END:X-DL-ITEM

That all makes sense, but what's that "DATA" line?

That's a reference to another contact. In this case, it happens to be from the GAL (Global Address List) and it is referencing the Public Contacts folder, specifically file 00000003.eml in that folder:

* 00000003.eml in Public Contacts Folder * 

Subject: Robert Lawrence
Date: Tue, 30 Apr 2013 15:15:06 -0400
Content-Type: text/vcard; charset="utf-8"
Content-Transfer-Encoding: 8bit

BEGIN:VCARD
VERSION:3.0
PRODID:-//kerio.com/Contacts//NONSGML v1.0//EN
EMAIL;TYPE=PREF:robert@aplawrence.com
N:Lawrence;Robert;;;
FN:Robert Lawrence
CATEGORIES:Global Address List
X-SYNCHRONIZED-FROM-GAL:true
UID:a5f72a5b-2264-4ca7-a4f5-8e09294ed86f
X-JABBER:robert@aplawrence.com
END:VCARD

How did I know it would be in the Public Contacts folder? Because inside "status.fld" in that folder is this:

Gfd3ab4b6-4fa0-460a-a14c-2eb13651e369

Look familiar? Compare it to

DATA:fd3ab4b6-4fa0-460a-a14c-2eb13651e369/00000003

See how the TYPE:REFERENCE works now? The DATA line points it to a folder and a specific contact. It's the same thing for the AAGroup entry - that's 00000011.eml in the Public Contacts.

What about "Contact Group One"? It's a LISTREFERENCE, which means it's pointing at another Contact Group and its DATA tells us it is in the folder with id e14df63c-e49e-45dc-ae98-976e6ff66fca and the file will be 00000198.eml. Where's e14df63c-e49e-45dc-ae98-976e6ff66fca ? That's the same Contacts folder that this Group is stored in; it's status.fld has Ge14df63c-e49e-45dc-ae98-976e6ff66fca in it.

Finally, look at that "nosuchperson" section. There's no DATA line for that, so it didn't come from an existing contact: it was just typed in.

So we see now how we can add items to a Contact Group - we find the members in the Public Contacts if we can and if we can't, we just put them in directly. That part is simple enough.

But what about the first part of the contents?

BEGIN:VCARD
VERSION:3.0
PRODID:-//kerio.com/Contacts//NONSGML v1.0//EN
X-DL:TRUE
FN:Nested Contact Group
UID:6596500c-5a9d-4788-8aa4-3f7e2ee324c3

What is that "UID:6596500c-5a9d-4788-8aa4-3f7e2ee324c3"?

That's a UUID. On Linux, you can generate one with

# uuid
b3d7bd76-d031-11e2-a82d-effcd23bafde

On Mac OS X, use

$ uuidgen
0DF91624-4C8C-477E-B0BA-5694769EBBF1

It's uuidgen on Windows also. You'll need a UUID for every Contact Group you create manually. There are also library routines for creating uuid's, but I'll use the command line versions here.

I took part of the code from my "Exposing hidden data to users" script. The script is fairly long and a bit clumsy, but I hope the logic is easy to follow.

Changes you'll need to make are marked in red.

Note that this particular script is unusually intrusive and could break things. PLEASE UNDERSTAND IT BEFORE YOU RUN IT. If possible, you should run this on copied data first to be sure it is doing what you want. Obviously you'd need to change some paths to do that.

#!/usr/bin/perl
$domainpath="/opt/kerio/mailserver/store/mail";
$domain="aplawrence.com";
open(I,"/opt/kerio/mailserver/users.cfg");
@stuff=<I>;
# These are the groups we want to change into Contact Groups
@groups=("Planning","Sales");
# Get Public Contacts uuid
 chdir("$domainpath/$domain/#public/Contacts");
 open(U,,"<:crlf","status.fld");
 @U=<U>;
 # uuid is second line and begins with "G"
 $cuid=$U[1];
 $cuid=~s/^.//;
 chomp $cuid;
 chdir("#msgs");
 $first_file_num=100;
 # Set $first_file_num above last currently in Public Contacts
 # Danger here:  NO ADDING OF PUBLIC CONTACTS WHILE RUNNING THIS!
 # If you can't guarantee that, SHUT THE SERVER DOWN!
 $newcontact=sprintf("%0.8x",$first_file_num);
#  Get group members
 foreach (@groups) {
   $group=$_;
   $uuid=`uuid`;
   chomp $uuid;
   while (-e "$newcontact.eml") {
     $newcontact=sprintf("%0.8x",$first_file_num);
     $first_file_num++;
   }
print "Writing $newcontact.eml for $group\n";
open(O,">:crlf","$newcontact.eml");
print O <<EOF;
Subject: $group Contact Group D-List
Date: Fri, 7 Jun 2013 08:20:40 -0400
Content-Type: text/x-vcard-dl; charset="utf-8"
Content-Transfer-Encoding: 8bit

BEGIN:VCARD
VERSION:3.0
PRODID:-//kerio.com/Contacts//NONSGML v1.0//EN
X-DL:TRUE
FN: $group Contact Group D-List
UID:$uuid
EOF
   
   $group=$_;
   foreach (@stuff) {
     chomp;
     $start=1 if /<list name="User">/; 
     $start=0 if /<.list>/;
      next if not $start;
      $user=$_ if /<variable name="Name">/;
      $user=~ s/<variable name="Name">//;
      $user=~ s/<.*//;
      $user=~ s/\s+//g;
      if (/<variable name="Groups">$group/) {
         # now find it in GAL Contacts
         chdir("$domainpath/$domain/#public/Contacts/#msgs");
         $found=0;
         foreach(<*.eml>){
           $emlfile=$_;
            $emlfile=~s/.eml//;
            chomp $emlfile;
            open(I, "<:crlf","$_");
            @cfile=<I>;
            close I;
            $fname="";
            foreach (@cfile) {
              $fname=$_ if /^FN:/;
              $found=1 if /^EMAIL.*$user\@$domain/;
             }
         last if $found;
         }
       $fname=~s/^FN://;
       chomp $fname;
print O <<EOF;
BEGIN:X-DL-ITEM
TYPE:REFERENCE
NAME:$fname
EMAIL:$user\@$domain
DATA:$cuid/$emlfile
END:X-DL-ITEM
EOF
      } 
   }
 print O "END:VCARD\n";
 close O;
 }
 chdir("$domainpath/$domain/#public/Contacts");
 rename("index.fld", "index.bad");
 # A server restart will fix things up quickly, but otherwise it may take some time

The hard part here is convincing Connect to reindex the Public Contacts. We rename "index.fld" in the script, but it can be a while before anything gets to there to fix it. You can hasten it with a server restart if you are impatient or use this.

After you are satisified that everything is as it should be, you can delete the Mail Groups and start using these.

Comments: Click Here.

Want to showcase your product to our audience? Check our advertising options.

Many of the products and books I review are things I purchased for my own use. Some were given to me specifically for the purpose of reviewing them.

Samepage - Redefining how people create and share information

Kerio Mail Server, Firewall and more