Setting up email in Linux can be a daunting and confusing task - that's for sure. Although there seems to be plenty of down-and-dirty documentation for each little tool, it can be VERY difficult to get the big picture and see how everything fits together. I must say that this is not really the fault of Linux, but rather the fault of "email" in general. It's a jungle out there.

In this document I'll *try* to give a survey of the main types of emailing setups. For the uninitiated and pseudo-initiated alike, it pays to just read the whole document through (without skipping sections). Many of the vocab words that we'll need I'll introduce in the first section. A quick warning: whenever I mention a tcp or udp port number, you should be thinking to yourself FIREWALL!!! Configure your FIREWALL to allow these ports!!!

Type 1: NOT HOSTING YOUR OWN DOMAIN
There are two main classifications of email systems. The first is very likely what you have: you don't have your own domain, or maybe you have your own domain but you are paying somebody ELSE to host it. In this case somebody ELSE gave you an email address, and you are just trying to "check your email". This type of system can be as easy or as hard as you like, depending upon your preference.

First, a little theory about how email works. Let's suppose your friend has an email address jack@hisdomain.com and you have an email address at jill@yourdomain.com. Then, whenever he writes an email to you, the mail server that is "in charge" of hisdomain.com contacts the email server that is "in charge" of yourdomain.com and sends the message on over. More precisely, there are complicated programs on each server called Mail Transport Agents (MTA) that talk to each other using a language called Simple Mail Transport Protocol (SMTP) - and it's always over the standard port 25.

Mail Transport Agent (MTA)
The MTA is really the HEART of the email "system". In fact, I wouldn't consider anything WITHOUT an MTA to be dubbed an "email system". It's the job of the MTA to receive email on port 25, process, and deliver locally to the appropriate user account - perhaps sending the mail through spam and/or virus filters first (there can be a whole BUNCH of little programs mixed up into this mess). It's also the job of the MTA to SEND emails out on port 25 to an MTA on a remote machine.

Some examples of MTAs are packages like Exim4, Sendmail (the original, but a bit outdated if you ask me), Qmail, Postfix, and Courier. By "deliver locally" I mean that the MTA puts the email into the correct user's "INBOX". Now this INBOX can come in two formats:

Mbox format. The older format is called the "mbox" format - this essentially collects EVERY email to a local user in ONE GIANT FILE. For example, Exim4 defaults to using the "mbox" format. When Exim4 receives email on port 25, it first gathers everything up into the /var/spool/exim4 directory. This directory is not your concern right now! Consider it a temporary playground for Exim4.

Once Exim4 is finished messing around with the email, it appends the email message (and attachments, too) to the file

/var/spool/mail/jill

This is your INBOX (in mbox format)! It is important to remember that the other email folders with which you are probably familiar (like SENT folder, or DRAFTS, or TRASH, etc) have NOTHING to do with your MTA or your INBOX. Those other folders are completely invented for convenience by your particular email *client* (the Mail User Agent - MUA - more on that later), and they are usually stored elsewhere.

Maildir format. Other MTAs, like Qmail or Courier, use a more modern INBOX format called "maildir" (Exim4 can also use the maildir format if configured appropriately - and we WILL). The differences between "mbox" and "maildir" are two-fold:

1) the huge monolithic INBOX file is now split into a lot of different files, and

2) the INBOX is now usually stored in the user's home directory. So, for example, the INBOX for user jill might be located in the directory

/home/jill/Maildir

So in this case the MTA delivers mail straight into the user's home directory!!!

Remote Access
The software that YOU see is called a Mail User Agent (MUA) or "mail client". It's job is to provide access to your local INBOX, and it usually provides other convenient folders like "SENT" or "DRAFTS". The details of these *other* folders depend HEAVILY on the MUA that you choose. Sometimes, if you mix and match them, then you can get strange consequences. The MUA also allows you to write and SEND emails. To do this, the MUA contacts the local MTA using the "sendmail" command and hands over the email for delivery. You don't need to know about the "sendmail" command - this is pretty much internal - and historical because the original MTA was (is) called "Sendmail".

We'll learn more about the "ins and outs" of MUA's later. The problem for you now is that YOU MIGHT NOT BE LOGGED IN LOCALLY. In other words, while your email is sitting safely on the server hosting "yourdomain.com", you aren't logged into it! Maybe you're at home, or whatever. You'll never access your INBOX that way! Now it is possible to log into the server and use the MUA provided there. For example, you could:

1) SSH in and check your mail with one of those text-based MUA's (such as "Mutt) that I haven't discussed yet, or

2) hopefully your server provides access via the Web (so-called "Webmail").

The beauty of both of these methods CANNOT be underestimated: both are secure!!! SSH kicks ass in general for security, and Webmail conversations take place over SSL (Secure Socket Layer) or TLS (OpenSource and more flexible than SSL). Hence, your passwords DON'T get left out in the open - in so-called cleartext - and neither are your emails themselves (although your emails were probably transmitted in "cleartext" from hisdomain.com to yourdomain.com IN THE BEGINNING, so that's not so important). The VERY important thing here is that your username and password info isn't broadcast where people can easily eavesdrop.

The other nice thing is that these methods require ZERO setup - and they're very reliable. In FACT, I ONLY use Webmail. It never changes, it never screws up, and I never lose emails.

In contrast, many/most people (i.e. your clients) don't want this. Instead they want an MUA (now installed on their home computer) to automatically connect to the server and somehow automatically download the INBOX. Then they can work offline, etc etc etc.

Here's the rub: this "somehow" method is OFTEN COMPLETELY INSECURE, and can cause serious data loss if care is not taken to do the "right" thing and somebody's hard drive crashes.

Nevertheless, here's how to do it. This is where the Mail User Agent (MUA) makes an appearance AGAIN, except now it operates in a different mode. Previously the MUA was installed on the server itself and was directly accessing your INBOX. Now the MUA will be installed on your home computer, and it will access your INBOX via logging into your server using a special set of protocols called POP/IMAP (incoming emails) and Smarthost (outgoing emails). Most MUAs like Mozilla Thunderbird easily work in this mode.

POP and IMAP
The POP and IMAP protocols are two different methods by which a client can download the INBOX (POP is older). The actual behavior using these protocols seems to vary WIDELY depending upon the mail CLIENT. For example, the standard Mozilla Thunderbird client behaves differently depending upon POP or IMAP, whereas with the "fetchmail" client (as we'll see later) POP and IMAP behave rather similarly.

For example, suppose that you are using Mozilla Thunderbird to access your mail via POP (to a POP server like "pop.sbcglobal.yahoo.com"). The standard Thunderbird+POP behavior is to *download* INBOX messages locally to the home computer and then DELETE them from the server! That's not good - what if you want to check your email from a different computer?!?!?! What if your harddrive CRASHES?!?!?!

I cannot think of a single instance where such behavior would be desired - unfortunately this is a Thunderbird default (most clients using POP act this way by default, in fact). This is why I usually direct people to use IMAP instead - POP is too easy to screw up (and once you screw up the first time, all of those emails are GONE from the server).

If you want POP to leave the messages on the server then go into "Edit->Account Settings->[your account]->Server Settings". You should click "Leave messages on server" and then click "Until I delete them or move them from Inbox". DO NOT check the "Delete after ?? days" option unless you know what this does. (These settings must be done BEFORE you check your mail *the first time*.) Note that, on the server, your mail will be changed from "unread" to "read", but at least a copy will stay on the server UNTIL you delete it or move it out of your INBOX.

If your ISP is complaining that your email account it taking up too much diskspace on their server then you should MANUALLY move some messages from your INBOX to some Local Folder - this will take them off of the server and keep them ONLY on your home computer. This solves the problem in a systematic fashion (so at least you know what is only locally copied, and what is still on the server).

One advantage of using POP (over IMAP) is that even IF mail is deleted from the server (perhaps your ISP screws up, whatever) then your INBOX at home is unaffected (because you downloaded a local copy). This can be very useful if you have one of those ISPs that changes their email system every two weeks and constantly purges people's INBOXes (grrrrrr). IMAP, on the other hand, does not have this advantage (emails gone from the server are gone from home, too).

So far so good. Standard POP connections occur on port 110, and are TOTALLY INSECURE! Your username and password are cast into the open with no encryption at all!!! However, people do it everyday... hey, it's only your bank account, right? I wish that I could show you here how easy it is to sniff your information, but there's no room here. Needless to say, a goldmine can easily be found in airports, coffee shops, and conferences where large numbers of people are logging into their emails constantly. There's nothing to stop a creep from just spending a few hours collecting data and then going to town on everybody.

Fortunately, there IS a way to clean this up. IF your server provides it, there's POP over SSL (aka POPS), which takes place over the standard port 995. What this does is it opens up an SSL encrypted tunnel to the server, then the standard POP session continues over this encrypted port 995. In Mozilla Thunderbird at the same panel you can click "Use secure connection (SSL)". Unfortunately not every ISP has moved passed the 1950s when everybody was talking about lollipops and ponies, so this might not be provided to you! You should complain - really.

Now there's one other possibility. The usual POP authentication (log in) takes place using the so-called AUTH LOGIN (worst) or AUTH PLAIN (still bad) protocols. This isn't an issue really if the channel is encrypted using SSL as above. Sometimes it's not, but instead your server MIGHT provide "secure POP authentication" using instead the CRAM-MD5 or Kerberos authentication procedures. This means that your username/password information is encrypted, but your emails themselves will be sent in clear text. That's usually fine, too, since your emails will probably be heading out in clear text eventually anyway. I *think* this "secure authentication" for POP is called APOP. You'll need to figure out for yourself if your server expects this (most don't). Since mine doesn't, in Thunderbird I left the "Use secure authentication" option UNCHECKED.

Obviously some good ISPs might combine both SSL and secure authentication for added security. Mix and match... mix and match.

IMAP: IMAP offers the better choice for *most* users because messages are never DELETED automatically from the server (except when using "fetchmail" - see below). Instead, if you set up a connection via IMAP then messages are left on the server until you explicitly DELETE them or MOVE THEM TO A LOCAL FOLDER! That's the behavior that I expect. The only disadvantage is the following: you won't get a local copy of your email - if your ISP server crashes then your emails are gone (there is an option in Thunderbird to make IMAP email available while offline, which pretty much gives you a local copy, but as soon as you log in the next time Thunderbird will see that all of your emails have been deleted from the server and will then update itself by deleting all of its LOCAL COPIES!!! OUCH!).

Usually it's either POP or IMAP, but sometimes both... so you're stuck with what you get. You'll usually be able to tell by the name of the server, e.g. "imap.sbcglobal.yahoo.com".

Now IMAP comes with the same security flaws that plague POP. Standard IMAP traffic goes over port 143 in CLEAR TEXT! Like POP the fix is IMAP over SSL (aka IMAPS) over port 993, if offered by your ISP.

Also like POP there is an "secure authenticated IMAP" to use CRAM-MD5 or Kerberos encrypted authentication, and obviously this can be teamed up with SSL for double protection. My ISP doesn't offer this, so in Thunderbird I left "Use secure authentication" UNCHECKED.

Smarthost
Now comes the part where you need to SEND emails. This is a little different. If you think about it, when your friend SENDS YOU an email from hisdomain.com to yourdomain.com, there's really no way to do it other than what I just described. The problem is that hisdomain.com has no idea where your home computer is, but it CAN find yourdomain.com by doing a DNS "MX record" lookup to find the server.

Conversely, if you SEND email FROM your home computer no such limitation exists in principle. When you send an email TO your friend jack@hisdomain.com then your home computer could just look up the MX record for hisdomain.com and your mail client (MUA) could just pretend to be an MTA and send the email on through (or you could set up your own little MTA and have it negotiate the connection). In practice this doesn't work very well because most respectable domains won't accept an SMTP connection from your crappy IP address. It knows, using blacklists, that your IP address is just some DSL or cable modem - not a permanent static domain - and it won't accept your traffic. We'll run into this issue again later EVEN IF WE HOST OUR OWN DOMAIN (unless we're willing to fork out the cash for a good static connection).

If you think about it, this is probably for the better. If anybody could send email to any server then the spam/virus problem would probably grow exponentially. In fact, many DSL and cable modem ISPs actually BLOCK your outgoing port 25 so that you can't even TRY! This is a REAL pain in the butt.

The solution is called Smarthost. Basically this is like POP/IMAP in reverse. This time you will SEND your email to YOUR server. The idea is that then your ISP will relay the message to the appropriate domain. Other domains WILL accept email from your ISP's mailserver since it has a "respectable name" - i.e. a static IP address. For some reason these "relays" are always called "SMTP" servers, but really they should be called "does SMTP for you" servers. An example is "smtp.sbcglobal.yahoo.com".

The question arises: how does the Smarthost server know to relay messages for YOU? Obviously they can't trust just anybody. There are two answers:

1) Sometimes the Smarthost is configured to accept all email from a certain range of IP addresses (after all, your ISP knows its customers).

2) More often your Smarthost will use "SMTP authentication" - i.e. logging IN!

Unfortunately, any time "authentication" is necessary then you are setting yourself up for username/password theft. like POP and IMAP, regular vanilla Smarthost traffic is NOT SECURE (but it happens every day)! The regular traffic occurs over port 25. There are 3 fixes:

1) Tunnel Smarthost traffic over SSL (port 465)

2) Tunnel Smarthost traffic over TLS - big brother to SSL (port 25 reused)

3) Use some form of SECURE authentication like CRAM-MD5 or Kerberos.

Obviously (3) can be combined with either (1) or (2)

Application: An INSANE SETUP with your OWN MTA
Let's apply these basic concepts, except let's make a REAL email system with your own MTA (you don't have to do this). It IS possible to have a FULL EMAIL SYSTEM on your computer but still use POP/IMAP and Smarthost to actually receive/transmit the mail. Why would anybody want to bother setting up a complicated MTA when it's pretty trivial to set up Thunderbird or something like it? Pure insanity, REALLY!

Actually, it's fun to set your own little mail server - good practice for later on if you buy your own domain and set up a server for real. The better answer is that many people like to bring to bear the full power of spam killing, antivirus, and filtering capabilities provided by mail servers.

Installing Exim4
The standard MTA that comes with Debian is called Exim4. It's rather good, and I must say that I had it set up in a couple of hours (although the more advanced stuff took several days). I should mention that I worked on Courier for two days and never did get the Smarthost client to login to my ISPs Smarthost server, so I ditched it and went for Exim4.

BEFORE WE BEGIN If you already have a version of exim installed then you may want to consider nuking the config files in /etc/exim4 - otherwise you may be missing some of the more recent advances like ready-made configurations for antivirus and antispam plugins. This guide covers Exim4 4.60-4.

Installation is easy, almost trivial. In Debian I installed the package "exim4-daemon-heavy". I could've just installed the package "exim4", but this would have installed a cut-down "light" version of the daemon with less capabilities. I want it all, so there you go. After that I went ahead and installed the package "exim4" as well.

The configuration is VERY friendly (uses DEBCONF). This configuration can always be re-run by using

dpkg-reconfigure exim4-config

Question 1 concerns whether you want to use the newer "split file" scheme or the old monolithic config file scheme. I chose the older one because I couldn't figure out where anything was with all of those files spread everywhere. Actually, I dislike the Exim4 config file system - it's not the usual "edit the config file and restart the daemon" routine. Instead, if you edit the configuration then you'll need to run

update-exim4.conf

This generates the REAL configuration file (located in /var/lib/exim4). Then you'll need to restart the daemon with "/etc/init.d/exim4 restart".

Back to business, Question 2 asks about your general setup. Unless you have a full domain WITH a static IP address AND good enough "reputation" to send emails then you should choose

mail sent by Smarthost; received via SMTP or fetchmail

We already talked about sending via Smarthost. You'll need Smarthost if you don't have your own domain, or if you have your own domain but you are hosting it from your DSL/cable modem. RECEIVING via SMTP is much less strict - you merely need a server of your own. This also means that the "MX record" for yourdomain.com must point to the IP address of YOUR COMPUTER! You don't really need a good reputation for servers to send TO YOUR DOMAIN, so you can RECEIVE as a full-fledged domain if you have one, even if you can't SEND by yourself!

On the other hand, if you DON'T host your own domain then you'll need to use POP or IMAP to download emails from the server that hosts your email account. You can still select the "receive via SMTP", but nothing will be sent to you. This is where the "fetchmail" program comes in. Fetchmail acts like a POP or IMAP client, downloads the relevant emails from your remote account, and then gives them over to your local MTA for local delivery. More about that later.

Question 3 asks about the "mailname.com" for your server. Any message sent from the user "jill" will be REALLY sent as "jill@mailname.com" (this can be overridden on a per-user basis by editing /etc/email-addresses, and users can override this themselves). You can use localhost.localdomain if you want, or you can make anything up that you like (BUT DON'T). There is no rule that says that the email domain needs to be valid. In fact, in the old days you could put in any bogus name that you want. You can even send emails in SOMEBODY else's name!

HOWEVER IN PRACTICE nowadays no mailserver will accept an email from "jill@localhost.localdomain", and most mailservers will drop an email from "jill@mailname.com" if "mailname.com" is not an ACTUAL domain. That doesn't stop you from using somebody else's domain, except that it's not a good thing to do (for a variety of reasons, including local delivery problems). Here's my advice: if you own your own domain then use that, if you don't then leave it as "localhost.localdomain" and we'll mask it later for each user separately.

Question 4 asks on what interfaces Exim4 should listen for incoming SMTP connections. If you are going to use Fetchmail to get your mails from a remote server then you should only use 127.0.0.1 (just listen to loopback - Fetchmail downloads the emails and sends them to your MTA using SMTP at 127.0.0.1). If you are actually going to receive emails AS your own domain then leave it blank (listen on all devices).

Question 5 asks for a list of domain names for which your computer will consider itself the "final destination". Only emails addressed to users at these domains will be accepted (however see below about setting up "virtual domains"). Everything else will be bounced. So if you're expecting people to receive emails to

user@yourdomain.com

then you should put "yourdomain.com" here (again, unless you are using virtual domains!). You should at least have the mailname (your answer to question 3) listed here.

Question 6 asks about those networks for which YOU will agree to relay email (without authentication). This just means that you might want your computer to act as a Smarthost *server* for OTHER computers (this has nothing to do with the fact that your computer will be a Smarthost CLIENT logging into the Smarthost server provided by your ISP - see the difference?). You probably don't want this unless you're setting up a whole network and domain of "trusted" computers. Don't just relay for anybody - that's VERY bad. In my case I am a Smarthost server, but I'm going to force my users to login using Smarthost/SMTP authentication, so I left this entry blank. (p.s. maybe I've used the "Smarthost" terminology poorly here? Maybe "relay" is more proper since "Smarthost" probably MEANS "relay+authentication". Who cares?

Question 7 asks for the name of the Smarthost server that you want to use. In my case I'm an SBC DSL customer so I'm forced to use "smtp.sbcglobal.yahoo.com". This worked for me, but you should NOTE that this name is actually an alias for "smtp-sbc.mail.yahoo.com", which in turn is an alias for the REAL mail server "smtp.sbc.mail.yahoo4.akadns.net". Sometimes you MUST use the real name. You can find this by using the command

host smtp.sbcglobal.yahoo.com

Question 8 asks about "Hiding" your OUTGOING mailname. I answer NO since this option will override any mailname that you have specified elsewhere (aliases, user-defined settings, etc).

Question 9 involves minimizing DNS queries - that's up to you.

Manual configuration of Exim4
You're NOT DONE YET!!! Now it's time to tweak some settings in Exim4. First, I'm using the "Maildir" format for my mailboxes, so to enable this you need to add the line

dc_localdelivery=maildir_home

to the file "/etc/exim4/update-exim4.conf.conf". Next, if you're logging into the remote Smarthost (you probably are) then you should put the Smarthost name, your username, and password in "/etc/exim4/passwd.client" - the format in there is obvious. Don't forget to change the permissions of that file to 640! There's sensitive password info in there that shouldn't be available to your users (a note - this file should be owned by user: root,group: Debian-exim).

As already mentioned, there are many Smarthost setups out there. The following describes how to set up your Smarthost *client* in 3 different situations (none of these uses "secure authentication" like CRAM-MD5 or Kerberos):

1) Smarthost over TLS on port 25
2) Smarthost in CLEAR TEXT
3) Smarthost over SSL on port 465

Smarthost client over TLS
IF your ISP's Smarthost server provides TLS then *I THINK* that you are done. Exim4 seems to default to using TLS, but I haven't actually tested this yet (I will someday soon... believe me).

Smarthost client in CLEAR TEXT, aka riding bareback

Now, unfortunately in my case my Smarthost "smtp.sbcglobal.yahoo.com" requires SMTP authentication (logging in), but it doesn't even use SSL/TLS. Furthermore, it doesn't use any of the "somewhat ok" encrypted username/password formats like CRAM-MD5, but instead it uses "PLAIN" which isn't really any better than the worst: "LOGIN". So my username/password is being broadcast out in the open (don't worry, I don't actually use this system. I just set this up for fun, then decommissioned it). If you are in this situation then don't tell anyone. If you still want to be silly then you can do it, but Exim4 tries to protect you from doing this accidentally. You need to explicitly tell Exim4 to allow you to broadcast your info in the clear.

Go into "/etc/exim4/exim4.conf.template" and look for the stanza (near the end) that starts like:

plain:
  driver = plaintext
  public_name = PLAIN
  ...

Smarthost client over SSL on port 465
The STRANGE thing here is that Exim4 doesn't seem to support SSL natively. Apparently it's TLS or nothing for Exim4. I NEED my Smarthost client over SSL, so we'll be forced to trick Exim4 into using an SSL tunnel. I found a good guide at http://www.technovelty.org/linux/tips, but I'm going to CHANGE SEVERAL THINGS!!!. First, I'm going to use the package "stunnel4" rather than "stunnel", so install the package "stunnel4".

The idea is that we're going to use "stunnel4" to set up some SSL tunnel(s). A warning: I'm setting up two tunnels here - one for my Smarthost *client* to connect to a remote host, and one IN ANTICIPATION of a Smarthost SERVER that you may want to set up later. I'll leave you to figure out what does what. Here's the strategy for now: Stunnel4 will listen on the *local* port 464 (that's right... I said 464, NOT 465 - I'm trying to avoid conflicts here) and be configured to login to the ACTUAL remote Smarthost whenever it receives traffic. That will be taken care of with this script "/etc/init.d/ssmtp-tunnel":

#! /bin/sh -e

case "$1" in
  start)
    echo -n "Starting ssmtp tunnels "
    /usr/bin/stunnel4 /etc/exim4/stunnelin.conf
    /usr/bin/stunnel4 /etc/exim4/stunnelout.conf
    echo "stunnel."
    ;;
  stop)
    echo -n "Stopping ssmtp tunnel "
    killall -9 stunnel4
    echo "stunnel."
    ;;
  restart)
    $0 stop
    sleep 1
    $0 start
    ;;
  *)
    echo "Usage: /etc/init.d/ssmtp-tunnel {start|stop|restart|reload|force-reload}"
    exit 1
esac

exit 0

You can load this into your startup scripts by running "update-rc.d ssmtp-tunnel defaults". Obviously you're going to need the configuration files "/etc/exim4/stunnelin.conf" and "/etc/exim4/stunnelout.conf", so here they are

# /etc/exim4/stunnelin.conf
client=no
cert=/etc/courier/imapd.pem

[smtp]
accept=465
connect=localhost:25

# /etc/exim4/stunnelout.conf
client=yes

[smtp]
accept=localhost:464
connect=remotesmarthost.com:465

Now the other necessary changes are easy... in "/etc/exim4/update-exim4.conf.conf" you should change your Smarthost to "localhost" like so

dc_smarthost='localhost'

and in "/etc/exim4/exim4.conf.template" you'll need to force Exim4 to allow you to send passwords in the clear like we did in the previous subsection (although secretly this is all going over an SSL tunnel - Exim4 just doesn't know it). So add the line

AUTH_CLIENT_ALLOW_NOTLS_PASSWORDS = true

like we did before, and tell Exim4 to go through port 464 by adding these entries in the "remote_smtp_smarthost:" stanza

hosts_avoid_tls = localhost
hosts_require_auth = localhost
port = 464

In the "smarthost:" stanza add the entry

self = send

Finally enable these changes by issuing

update-exim4.conf
/etc/init.d/exim4 restart
/etc/init.d/ssmtp-tunnel start
and you're DONE!

Important note Remember your "/etc/exim4/passwd.client" file! As far as Exim4 is concerned, the Smarthost server is localhost. You MUST use "*" as your server NAME in there!!!

Your first MUA - using Mutt
I'm not a Mutt expert. I think that it's a good email client - text based and fast, but the setup is a little ridiculous. I'm not going to spend a week trying to figure out all of the ins and outs of it, so here goes. It's important to have SOMETHING to send email, so install the package "mutt".

Now, as a user you cannot just charge ahead and use Mutt without a little setup. First, you will actually need to manually create your Maildir INBOX before you can send/receive anything. So install the package "courier-base". This actually installs part of the Courier MTA, but that's OK. All that we want for now is the command "maildirmake".

Login as the relevant user and run

maildirmake ~/Maildir

This creates an INBOX in that user's Maildir directory. Now you're almost ready to get somewhere (note: many sysadmins put a Maildir directory in /etc/skel so that they don't have to do this for EVERY user).

The only piece left is a custom .muttrc in your home directory. This is necessary because Mutt expects to use the mbox-type INBOX and we want to use maildirs instead (for good reason - we'll see later). The following is a VERY barebones ~/.muttrc to do what you want

# Where is your INBOX?
set spoolfile=~/Maildir
set folder=~/Maildir
set mbox_type=Maildir

# We are going to set up a Sent, Drafts, and Trash folder.
# We might as well just store them as subdirectories
# ~/Maildir/.Drafts
# ~/Maildir/.Sent
# ~/Maildir/.Trash
# These folders will each have their own little "maildirs" format.
# This is a STANDARD place for the IMAP server to put its folders, too
# I'm looking for compatibility with IMAP so that if I use Webmail
# or login remotely using IMAP then the folders are preserved
#
# This mask is necessary so that when you press "c" to bring
# up your folder list these folders won't be hidden.  It's not pretty,
# but it works.
set mask="."

# Send folder
set record=~/Maildir/.Sent
set copy=yes

# Keep INBOX messages in INBOX - why move them unless you want to?
set move=no

# Drafts folder
set postponed=~/Maildir/.Drafts
set postpone=yes

# There is no built-in Trash command in Mutt
# This maps "d" to Trash and creates the trash folder
macro index d "=.Trash\n"
macro pager d "=.Trash\n"

# Set your "From" information
set from="user@yourdomain.com"
set use_from=yes
set use_envelope_from=yes

That's it! Start by trying to send an email to yourself or other users on the local machine (just use their usernames - the domain will be filled in automatically). Then try to send an honest email out to the world. The debugging log is in /var/log/exim4/mainlog. You can see which emails are still in the outgoing queue with the "mailq" command.

Fetchmail
Now it's time to figure out how to DOWNLOAD emails from a remote IMAP or POP server and inject them into your local mail system as though they were received the usual SMTP way. Fetchmail is your tool here. Now forget everything that I said above about how POP and IMAP differ. Fetchmail, for most things, treats them very much the same. By default, Fetchmail downloads messages and then deletes then off of the server. If you want to change that behavior, then you can use the "keep" keyword (see below) which will download messages but merely mark them as "read" on the server.

To start just install the package "fetchmail". Create a ~/.fetchmailrc file in the relevant user's home directory (the emails have to go to SOMEBODY, right!). In there you can put commands such as

defaults
set no bouncemail
set no spambounce
poll pop.sbcglobal.yahoo.com protocol pop3 username \
  "asdf@hotmail.com" password "YOURPASSWORD" keep

Obviously "asdf@hotmail.com" is the email account that you are downloading, and the "keep" command just says to not delete the files off the server. This is for POP... an IMAP script would be similar. The variations on this theme are endless. Just run the command "fetchmail" and watch those emails come on down (the "Envelope-to" address will be changed to whichever user invokes Fetchmail so the Exim4 knows where to deliver the mail)! Note that the POP default is to not download messages that have been marked as "read" on the server. You can change this with a "fetchall" keyword after "keep" above. To automate this you can put a "fetchmail" line in your crontab to be run every 10 minutes or so.

If your IMAP/POP server is expecting an SSL connection then you can use a script like the following (here I'll use IMAP on port 993 to demonstrate):

defaults
set no bouncemail
set no spambounce
poll imap.sbcglobal.yahoo.com protocol imap port 993 username \
  "asdf@hotmail.com" password "YOURPASSWORD" ssl keep

Note the "keep" keyword there, even for IMAP.

Where do you go now?
At this point you should have a functioning email system. Now what to do next? You'll want spam and virus filters. You may even want to set your machine up as a POP/IMAP/Smarthost server in its OWN RIGHT! A webmail interface would be nice, too. I'll show you how to do all of that in the next section.

Type #2: HOSTING YOUR OWN DOMAIN
If you host your own domain on your own server then the setup should be very similar to the "insane" setup above, except you won't need fetchmail to grab those emails off of another server because now YOU are hosting the domain (your server will RECEIVE emails directly via SMTP). Make sure that port 25 is open into your server! Make sure that your domain "MX record" points to YOUR SERVER!!! You'll need to go to your internet registry to do that (I'm using ZoneEdit and Dynamic DNS so I went there to do it). In question 4 above leave it blank (listen on all interfaces).

SENDING email from your own server is still a bit tricky. For example, although I'm hosting my own domain (on a computer in the middle of the forest, which explains the slow connection!), I only have a dynamic IP address (see my Dynamic DNS article for hints about how to set something like that up). This means that I shouldn't directly SEND emails to other email servers - instead I still am forced to use my ISP's Smarthost as discussed above (otherwise half of my outgoing email will be bounced back as untrusted). If you have a static IP address then you won't need to use your ISP's Smarthost/SMTP server as a relay... you can just send directly (make sure that the outgoing port 25 is open in your firewall! Also many ISP's block outgoing port 25, so you might have to make a special request for them to open it).

Setting up Courier IMAP and Squirrelmail
Now for some more fun. Maybe you want to set your machine up as an IMAP and Smarthost *server* (now YOU'RE the host, and so your users will expect these services). You'll probably want to set up a Webmail interface as well.

IMAP (over SSL) server using Courier
Fortunately, this is the really FUN part. You'll get a lot for your minimal efforts. Here I'm going to use the IMAP (over SSL) server provided as part of the Courier MTA. Yes, it's true - this system is half Exim4, half Courier. That doesn't matter. In fact, it started as a complete Courier system, but I couldn't get the Smarthost client to authenticate, so Exim4 it was. Now it's a big Frankenstein, but this is (I think) probably the most common configuration out there. Actually, there are A LOT of tools that make Courier a VERY good MTA. The IMAPS/POPS servers, the Courier authentication daemon - it's all REALLY good stuff.

To install Courier IMAP is almost trivial. Again, no need to reinvent the wheel. There's a good howto at http://edseek.com/~jasonb/articles/exim4_courier/courierimap.html.

After you've followed the instructions on that page, you should have an IMAPS server listening on port 993.

One quick note - under the hood all of this stuff uses the Courier authentication module "courier-authdaemon" to do the actual password checking. We'll use it again, even with Exim4! If you run into login problems then you can use the "courierauthtest" utility to test your authdaemon. I've noticed that most problems come from permissions issues. You should set the permissions as follows

chmod 775 /var/run/courier/authdaemon
chmod 777 /var/run/courier/authdaemon/socket

At this point we could really implement some security. You might want to have a look at the "secure authentication" modules for Courier IMAP (like CRAM-MD5), but to be honest I'm not freaked out about clear-text as long as the channel is encrypted (I know, I'm probably letting a lot hang out here).

Squirrelmail
Now that you have IMAP set up you can install the Squirrelmail webmail client. Personally, I'm a big fan of Squirrelmail. It is very easy to install, and I like the interface. Note that Courier supplies its own webmail client called "sqwebmail" - you won't even need Courier IMAP installed for that to work - just install the package "sqwebmail" and you'll have a fully-functioning webmail CLIENT! Cool, huh!

However, to be honest, I like the user interface to Squirrelmail a bit better, and since I already have the IMAP server installed for other reasons, I'm going to use it.

Start by making sure that you have PHP support included with your Apache server. In my case I'm still running Apache 1.3 so I installed the package "libapache-mod-php5". Now you're ready to rock: install the package "squirrelmail". After installing you will receive the instructions to run "squirrelmail-configure". I ran it, but I didn't actually change anything in there. Note that the "folders" are listed as INBOX.Sent, INBOX.Drafts, and INBOX.Trash - these will ACTUALLY be located in ~/Maildir/.Sent, ~/Maildir/.Drafts, and ~/Maildir/.Trash (think of INBOX as an alias for ~/Maildir), so you won't need to change the default.

Save the settings and exit.

Squirrelmail is set up, but Apache doesn't know about it. There's a handy little file called "/etc/squirrelmail/apache.conf" that gives some good ideas. This is the time when your interface with Apache might get a little sketchy. You see, you MUST have your Squirrelmail going over HTTPS on port 443 instead of regular HTTP on port 80, so you'd better make sure that the appropriate stanzas in your "/etc/apache/httpd.conf" are configured to use SSL. Don't just use the regular - that would leave passwords out in the clear!

In my case, I never could get Apache with the "libapache-mod-ssl" SSL module to work, so instead I use TWO different Apache webservers, BOTH RUNNING SIMULTANEOUSLY. One regular and one with SSL statically linked in! For my regular Apache webserver I am using the package "apache-perl" (which has perl statically linked in, too)... the configuration file there is in

/etc/apache-perl/httpd.conf

For SSL-enabled Apache I use the Debian package "apache-ssl". The relevant configuration file is located in

/etc/apache-ssl/httpd.conf

#SQUIRRELMAIL STUFF

Alias /squirrelmail /usr/share/squirrelmail

<Directory /usr/share/squirrelmail>
  php_flag register_globals off
  Options Indexes FollowSymLinks
  <IfModule mod_dir.c>
    DirectoryIndex index.php
  </IfModule>

  # access to configtest is limited by default to prevent information leak
  <Files configtest.php>
    order deny,allow
    deny from all
    allow from 127.0.0.1
  </Files;>
</Directory>

Notice what's going on here. Whenever a user directs their browser to "https://www.spencerstirling.com/squirrelmail" they should get the Squirrelmail login. Restart the Apache SSL webserver so the changes take effect.

Now, in the regular Apache webserver I like to have a "redirection" so that if somebody puts in http:// then the get redirected to https://. The following does the trick:

#SQUIRRELMAIL REDIRECTION STUFF

Alias /squirrelmail /usr/share/squirrelmail
<IfModule mod_rewrite.c>
    <Location /squirrelmail>
      RewriteEngine on
      RewriteCond %{HTTPS} !^on$ [NC]
      RewriteRule . https://%{HTTP_HOST}%{REQUEST_URI}  [L]
    </Location>
</IfModule>

Restart your regular Apache webserver and you're done.

Smarthost server setup in Exim4
Now it's time to set your machine up as a Smarthost SERVER (so others can relay mail through you). I'm not going to fight the path of least resistance - here I'll describe 3 setups (none of which use "secure authentication" modules like CRAM-MD5).

1) Smarthost in CLEAR TEXT over port 25
2) Smarthost over TLS on port 25
3) Smarthost over SSL on port 465

Smarthost server in CLEAR TEXT over port 25
Here a way to allow insecure connections:

First, you must tell Exim4 to allow people to login to your server on a clear channel. So add the line

AUTH_SERVER_ALLOW_NOTLS_PASSWORDS = true

in "/etc/exim4/exim4.conf.template" BEFORE it's referred to by other stanzas. You'll see where.

Now you'll need to tell Exim4 to use the Courier authentication daemon to do the password checking. After the place where you put the previous entry you'll see many stanzas with several different authentication methods available. Uncomment the following stanzas

plain_courier_authdaemon:
  ...
  
login_courier_authdaemon:
  ...

Now you can test your connection with a telnet to port 25 (remember to update-exim4.conf and restart exim daemon first) - consult http://qmail.jms1.net/test-auth.shtml for good information about that! You can test courier-authdaemon with the "courierauthtest" command if you are having problems authenticating.

Smarthost server over TLS on port 25
For a secure TLS connection first follow the steps in the previous subsection. Next, you'll need to start the TLS server by adding the entry

MAIN_TLS_ENABLE = true

BEFORE it's referred to by other stanzas.

Now you'll need to generate your TLS certificates. Use the following commands

cd /etc/exim4
/usr/share/doc/exim4-base/examples/exim-gencert

That's IT! Yes - I know. You ALREADY generated certificates for Courier IMAPS. These are for Exim4 Smarthost server. That's the price that we pay for using two packages. It's amazing that they coexist so well!

If you want to FORCE your users to use TLS then you should take out the "AUTH_SERVER_ALLOW_NOTLS_PASSWORDS = true" line that you added in the previous subsection.

Smarthost server over SSL on port 465
Now you MIGHT want to set up a Smarthost server over SSL on port 465. Why? In my case, it's because my ISP is filtering my traffic on port 25. For some reason regular SMTP traffic gets through, but TLS traffic is being filtered (*shrug*). Using an "stunnel4" hack I can just enable Smarthost over SSL and be done with it. By the way, if you want to know how to perform a port scan to see what is happening with YOUR firewall, try the command (from a remote machine)

nmap -sV yourdomain.com

First, set up Exim4 as a Smarthost server as described in the last two subsections - just make sure that Smarthost over CLEAR TEXT is enabled (you can also use TLS, giving you all three worlds AT ONCE! Cool, huh?).

Of course Exim4 doesn't handle SSL, so we'll be forced to use the same scheme already described when we were talking about setting up a Smarthost CLIENT over SSL. In fact, all of the relevant STunnel configuration was already done there - so there's nothing left to say!!! Stunnel should be listening on port 465 and directing traffic to port 25 locally (where Exim4 is listening).

Hosting MULTIPLE DOMAINS
Hosting multiple domains is possible. The only issue is that you need both incoming and outgoing aliases to a) determine how to route incoming emails to local users, and b) to determine the default OUTGOING email addresses for each user.

If you have a relatively simple setup (not 1000 different domains) then you can handle both of these rather easily.

Virtual Domains for INCOMING mail
Start by editing the file "/etc/exim4/domains.virtual" and add any domains that you want to "virtually" host, like so

hotmail.com
sbcglobal.net

@hotmail.com: user1
asdf@hotmail.com: user2
poiu@hotmail.com: user3

jimmy@sbcglobal.net: user1

You can actually forward any email ANYWHERE! Cool, huh?

Of course, you'll need to update "/etc/exim4/exim4.conf.template" as well. Look for the line "domainlist local_domains" and tack onto the end the entry ":lsearch;/etc/exim4/domains.virtual", for example:

domainlist local_domains = MAIN_LOCAL_DOMAINS:lsearch;/etc/exim4/domains.virtual

domains_virtual:
driver = redirect
data = ${lookup{$local_part@$domain}lsearch{/etc/exim4/aliases.virtual}}

domains_virtual_others:
driver = redirect
data = ${lookup{@$domain}lsearch{/etc/exim4/aliases.virtual}}

Update Exim4 and restart the daemon. Go outside and do a raindance... hopefully this works.

/etc/email-addresses for OUTGOING mail
Assigning a default email address on OUTGOING email is just as easy. If you don't set anything up then the email will originate from "user@mailname" (what you answered in question 3 during the Exim setup), although the users can always configure this themselves in their mail client.

Often neither of these options is going to be acceptable. Your users will SEAMLESSLY want their emails to come from THEIR domain - they don't want to even know about it. In this case you can manually set the default email address (on a per-user basis) by editing the file "/etc/email-addresses". The format in there is obvious.

Clamav Antivirus and Spamassassin
This is getting long, but let's press ahead! You'll want software that checks for viruses in those emails, and you'll want to kill spam! Clamav is your virus killer. It's free, it's good, and it updates ITSELF (using the "clamav-freshclam" package). Much of the following was shamelessly stolen from http://koivi.com/exim4-config/ (with whatever changes I needed).

Start by installing the following packages:

clamav
clamav-base
clamav-daemon
clamav-freshclam
clamav-testfiles
arj
unzoo
lha
unrar
spamassassin
spamc
sa-exim

I don't know why, but I needed to rerun "dpkg-reconfigure clamav-base" to get the socket to take effect - I think that the default is

/var/run/clamav/clamd.ctl

You'll need to add the "clamav" user to the "Debian-exim" group so that Exim4 can access this socket!!! Furthermore, you'll need to change ownership of the clamav directory:

chown Debian-exim:Debian-exim /var/run/clamav

In fact, to avoid problems I also run the clamav daemon as user "Debian-exim". Just edit "/etc/clamav/clamd.conf" and set

User Debian-exim

Now comes the easiest part of ALL!!! You need to edit "/etc/exim4/exim4.conf.template". First look for the antivirus and spam section. You should add/change the appropriate line to

av_scanner = clamd:/var/run/clamav/clamd.ctl

Note the correct socket. You should also uncomment the line

spamd_address = 127.0.0.1 783

Apparently Spamassassin runs as a daemon listening to port 783. The comments there instruct you to the "acl_check_data" stanza. Go there and uncomment the lines

   deny
     malware = *
     message = This message was detected as possible malware ($malware_name).

   warn
     spam = nobody
     message = X-Spam_score: $spam_score\n\
               X-Spam_score_int: $spam_score_int\n\
               X-Spam_bar: $spam_bar\n\
               X-Spam_report: $spam_report

  deny
    spam = nobody
    message = 550 Unrouteable address : User unknown\n\
              Write "postmaster" for questions

/etc/default/spamassassin

ENABLED=1

Now edit "/etc/spamassassin/local.cf" and add the following lines

trusted_networks 127.0.0.1
dns_available yes
skip_rbl_checks 1

Give yourself a little "update-exim4.conf" and "/etc/init.d/exim4 restart" and you'll be ON YOUR WAY! Note: I think this was once MUCH more difficult - it's pretty slick NOW!

Testing Clamav and Spamassassin
First, let's test Clamav. I had you install the "clamav-testfiles" package for a reason - in "/usr/share/clamav-testfiles" you'll see several "clam" files with various extensions. These are virus *signatures* - not VIRUSES!!! They'll trip the alarm, but cause no problem. You should email these to yourself from a different DOMAIN - Clamav doesn't check emails that originate internally.

If all goes well then you'll never get them. Check your /var/log/exim4/mainlog log - you should receive several warnings of "Malware" and rejected email. That's GOOD!

Now for Spamassassin. There are two files located in "/usr/share/doc/spamassassin/examples", namely

sample-spam.txt
sample-nonspam.txt

Transfer these files to an account in a different domain and then mail yourself back with

mail jill@yourdomain < sample-spam.txt
mail jill@yourdomain < sample-nonspam.txt

You'll receive both messages, but if all goes well you'll see a lot of spam warning messages in the header of the second email. These are directives flagging the email for further processing.

What further processing should we use? The simplest is to put a .forward file in the home directory of each user with the following entries:

# Exim filter
if  $h_X-Spam_report: CONTAINS "possible spam"
then
  save $home/Maildir/.Spam/
  finish
endif

It's pretty obvious what this does. You'll need to make each user a Spam folder with the command

maildirmake ~/Maildir/.Spam

From a sysadmin point-of-view, I always put all of the following items into "/etc/skel" so that when I create new users they automatically have these settings in place

/etc/skel/Maildir
/etc/skel/Maildir/.Spam
/etc/skel/.forward
/etc/skel/.muttrc

sa-learn --spam /home/fred/Maildir/.Spam/cur/*

This is pretty weak, I know, but I don't care! I won't claim to be a Spamassassin expert. Clamav seems to take care of itself, like owning a cat. The proper care and feeding of Spamassassin is more like a dog - always begging for attention and constantly having to take a piss.

BLACKLISTS
Aren't we done yet? Not quite. You'll want to use so-called DNSBLS blacklists (e.g. spamhaus.org), just like everybody else. This will reduce your spam A LOT! In Exim4 this is almost trivial. Go into "/etc/exim4/exim4.conf.template" and look for the stanza

  .ifdef CHECK_RCPT_IP_DNSBLS
  warn
    message = X-Warning: $sender_host_address is listed at $dnslist_domain ($dnslist_value: $dnslist_text)
    log_message = $sender_host_address is listed at $dnslist_domain ($dnslist_value: $dnslist_text)
    dnslists = CHECK_RCPT_IP_DNSBLS
  .endif

    message = 550 Unrouteable address : User unknown\n\
              Write "postmaster" for questions

To actually specify your DNSBLS services, you'll need to add a line like this

CHECK_RCPT_IP_DNSBLS = sbl-xbl.spamhaus.org:dnsbl.sorbs.net:combined.njabl.org:bl.spamcop.net:psbl.surriel.com

You should search for different DNSBLS services out there. In particular, it's important to block any dynamic IP addresses. Hell, you had to use smarthost, right?!?! Why should you accept mail (usually spam) from some other idiot on their DSL? Make them use Smarthost, too... at least their ISP can be held accountable then. It's an ongoing battle, for sure.

Note that you can start a "fake" SMTP session to see if a certain IP address will be rejected by issuing the following command

exim4 -bh [IP address]

ehlo assholespammer.com
MAIL FROM:somejerk@assholespammer.com
RCPT TO:jill@yourdomain.com

LOCAL BLACKLISTS (a Honeypot)
Honeypots are really cool. The strategy is this: make up an email address on your server that doesn't exist (and probably won't in the future), say "sosweet@example.com" (where "example.com" is your domain). Now purposefully PLACE this email address in HIDDEN places on your websites (I mean hidden from human viewers). I have a hidden link on every page! Yes, let it be harvested. The following link works for me:

<a href="mailto:sosweet@example.com"><font color="white">haha</font></a>

First you'll need to configure "/etc/exim4/exim4.conf.template" to set up a local filter. I stole these ideas directly from Mark Knox's page http://www.hardline.org/SpamHoneypot. Somewhere in that file (I put mine at the top of the "main/config-options" section you should put the following stanza:

# setup HONEYPOT filters (fake email addresses used as bait)
system_filter = "/etc/exim4/system.filter"
system_filter_user = Debian-exim
system_filter_group = Debian-exim
system_filter_pipe_transport = address_pipe

  deny
    message = sender IP address $sender_host_address is locally blacklisted here. If you think this is wrong, get in touch with postmaster
    !acl = acl_whitelist_local_deny
    hosts = ${if exists{CONFDIR/local_host_blacklist}\
                 {CONFDIR/local_host_blacklist}\
                 {}}

    message = 550 Unrouteable address : User unknown\n\
              Write "postmaster" for questions

Now you'll need to set up the filter itself. Create a file called "/etc/exim4/system.filter" and place the following lines in there (with the appropriate changes):

if $recipients contains "sosweet@example.com"
then
  pipe "/etc/exim4/blacklist-me $sender_host_address"
endif

#!/bin/sh
BLACKLIST=/etc/exim4/local_host_blacklist
BLACKLISTDATES=/etc/exim4/local_host_blacklist_dates

echo $* >> $BLACKLIST
echo $* `date +"%Y-%m-%d %H:%M:%S"` >> $BLACKLISTDATES

touch /etc/exim4/local_host_blacklist
touch /etc/exim4/local_host_blacklist_date

chown Debian-exim:Debian-exim /etc/exim4/local_host_blacklist
chown Debian-exim:Debian-exim /etc/exim4/local_host_blacklist_date

sosweet@example.com: :blackhole:

This page has been visited   times since March 15, 2006