help centre
For more info visit status.atmail.com

How can we help?


Search our knowledge base for answers to
common questions and latest updates.



My activities New request

Follow

multi server setup guide - installation and filesystem replication

Stewart -

PROBLEM
I'm not sure how to install atmail mail server in a multi server environment.

ENVIRONMENT

  • atmail on-premises mail server installations: current version

CAUSE
atmail has not yet been installed.

RESOLUTION

Before continuing with this guide, setup MySQL master-master replication or MariaDB master-slave replication first.

Installing atmail's First Instance

You can now install atmail. You will need to do this on only one of the servers first, followed by the NFS replication setup. For the purposes of this guide, we will use the server1.local as configured above (192.168.0.2).

 Mail Server Installation

Email Replication Across Servers

There are three methods for replicating email files across servers: NFS, GlusterFS and Unison. Instructions on how to implement your preferred replication format follow below

GlusterFS

GlusterFS is an open source clustered file system capable of scaling to several petabytes and handling thousands of clients. GlusterFS clusters together storage building blocks, aggregating disk and memory resources and managing data in a single global namespace.

A key advantage is the GlusterFS modular architecture that allows modules to be stacked to match user requirements. You can use GlusterFS to quickly configure a standalone server system and later expand the system as your needs grow. For more information, see: http://gluster.org

Note: There are two sub-tutorials - one for GlusterFS 3.0 and one for GlusterFS 2.0. We highly recommend that you use the GlusterFS 3.0 documentation.

For this tutorial, there will be two servers - server 1 (192.168.0.2) and server 2 (192.168.0.3). This tutorial, with its usage of yum, is primarily for CentOS 5.4. Feel free to use the equivalent package manager for your operating system.

GlusterFS 3.3

First, make sure that the fuse module is installed:

% modprobe fuse

Check that it's installed by running this command:

% dmesg | grep -i fuse

The above command should produce output similar to this:

% fuse init (API version 7.10)

Once the fuse module is installed, you are now ready to install GlusterFS 3.3. Note that this installation will require the deactivation of previous GlusterFS installations (i.e., 3.2 or older). Make sure your backups are in place.

On both machines, download the GlusterFS 3.3 source from: http://download.gluster.org/pub/gluster/glusterfs/LATEST/

Untar, install:

% tar xvfz glusterfs-3.3.0.tar.gz
% cd glusterfs-3.3.0
% ./configure && make && make install

The output of the end of the installation should be similar to:

FUSE client : yes
Infiniband verbs : yes
epoll IO multiplex : yes
argp-standalone : no
fusermount : no
readline : yes

Start the GlusterFS service on both machines:

% /etc/init.d/glusterd start

Once the service is started, run the following on server 1 (Where 192.168.0.3 is the other server):

% gluster peer probe 192.168.0.3

The output should be similar to:

% gluster peer probe 192.168.0.3
Probe successful

You can test the peer status further by executing:

% gluster peer status

Which should yield similar output to the following:

Number of Peers: 1
Hostname: 192.168.0.3
Uuid: f31c579d-01d0-4f70-8e96-c72f1c691167
State: Peer in Cluster (Connected)

Afterwards, create the GlusterFS volume on server 1. This command will also create the GlusterFS volume on server 2:

% gluster volume create atmail replica 2 transport tcp 192.168.0.2:/export/atmail/ server2:/export/atmail/

In the above command, replace the IP addresses with the IP addresses of your GlusterFS servers; replace /export/atmail/ with your preferred GlusterFS export directory.

Start the volume with this command:

% gluster volume start atmail
Starting atmail has been successful

Mount the directories afterward to your desired mount points. For this example, we will use the atmail users/ directory.

On server 1, mount the GlusterFS volume:

% mount -t glusterfs 192.168.0.2:/atmail /usr/local/atmail/users/

Do the same for server 2:

% mount -t glusterfs 192.168.0.3:/atmail /usr/local/atmail/users/

Create the tmp directory for server1:

% mkdir -p /usr/local/atmail/users/tmp/

Set a symlink to the /usr/local/atmail/webmail/tmp/ directory:

% ln -sf /usr/local/atmail/users/tmp /usr/local/atmail/webmail/tmp

Do the same for server 2:

% ln -sf /usr/local/atmail/users/tmp /usr/local/atmail/webmail/tmp

Congratulations. Your installation is now setup to use GlusterFS 3.3.0.

Unison As A File-Replication System

Unison allows two replicas of a collection of files and directories to be stored on different hosts (or different disks on the same host), modified separately, and then brought up to date by propagating the changes in each replica to the other. In other words, it can be used for two-way synchronization between your servers.

For illustrative purposes in this example, the Unison Master hostname will be called "master-foo-01"; the Client hostname will be called "slave-foo-02".

Installing Unison

To setup Unison on your machine, do the following for both machines. Download the dependency, OCaml from: http://caml.inria.fr/download.en.html

Install emacs (a dependency):

atmail@master-foo-01# yum install emacs

Unpack, install OCaml:

atmail@master-foo-01# tar xvfz ocaml-3.12.0.tar.gz
atmail@master-foo-01# cd ocaml-3.12.0
atmail@master-foo-01# ./configure && make world
atmail@master-foo-01# make opt
atmail@master-foo-01# make install

Download Unison from: http://www.cis.upenn.edu/~bcpierce/unison/download.html

Untar, install:

atmail@master-foo-01# make world opt
atmail@master-foo-01# sudo make install

Move the Unison binary to your preferred prefix. We suggest /usr/bin/:

atmail@master-foo-01# mv unison /usr/bin/unison
atmail@master-foo-01# chmod 755 /usr/bin/unison

Creating login keys for Unison

You will then need to set the keys, so the servers can automatically talk to each other. Create a SSH public key on the Master server:

atmail@master-foo-01# ssh-keygen -t rsa

This will produce a public key in /home/atmail/.ssh/id_rsa.pub. Copy the .pub file to the Slave machine:

atmail@master-foo-01# scp /home/atmail/.ssh/id_rsa.pub root@slave-foo-02:/home/atmail/.ssh/id_rsa.pub

Pipe the public key into a file called authorized_keys, in your .ssh directory.

atmail@slave-foo-02# cat /home/atmail/.ssh/id_rsa.pub >> authorized_keys

Make sure that the authorized_keys file, and all of the contents of the .ssh directory have 600 permissions:

atmail@slave-foo-02# chmod 600 /home/atmail/.ssh/*

Try to login, sans password, from the Master machine to the Client machine.

atmail@master-foo-01# ssh atmail@slave-foo-02

This should allow you to login.

Running Unison

Now, you can try to sync the differences between the users/ directory of the master and slave machines. From any of the machines, execute:

atmail@master-foo-01# unison -batch -auto /usr/local/atmail/users ssh://slave-foo-01//usr/local/atmail/users

The output should be similar to:

UNISON 2.40.63 finished propagating changes at 01:25:25.57 on 18 Apr 2011
Saving synchronizer state
Synchronization complete at 01:25:25  (xx items transferred, 0 skipped, 0 failed)

You will need to set this in cron. Create a file called /home/atmail/unison.sh. In the file:

if [ ! -f /home/atmail/unison.pid ];
then
touch /home/atmail/unison.pid
/usr/bin/unison -batch -auto /usr/local/atmail/users ssh://slave-foo-01//usr/local/atmail/users
echo "Unison process executing..."
rm /home/atmail/unison.pid
else
echo "Unison process still running"
fi

A Note about Unison preferences

Sometimes, Unison will find collisions with file creation and attributes. In this case, you will need to set the server were the changes are automatically synced from, in case of file collisions. This is referred to as the preferred server .

In this case, we will choose the local machine as the preference. You can use either copy; we generally recommend that you use the server that is utilized the most. We will change the above bash script to reflect this:
if [ ! -f /home/atmail/unison.pid ];
then
touch /home/atmail/unison.pid
/usr/bin/unison '''-prefer /usr/local/atmail/users''' -batch -auto /usr/local/atmail/users ssh://slave-foo-01//usr/local/atmail/users
echo "Unison process executing..."
rm /home/atmail/unison.pid
else
echo "Unison process still running"
fi

Set permissions:

chmod 755 /home/atmail/unison.sh

Add an entry to your /etc/crontab. It will look like:

01,10,20,30,40,50 * * * * atmail /home/atmail/unison.sh > /dev/null 2>&1

This will synchronize your /usr/local/atmail/users directory for both machines.

NFS Replication

NFS is another viable alternative. NFS allows a user on a client server to access files over a network in a manner similar to how local storage is accessed. One potential drawback is that there is a single point of failure. However, this can be minimized by splitting directories across two servers. The tutorial below will cover basic server-client NFS setup.
To get started, open up the /etc/exports file on the first server (server1.local). Add the following lines:

/usr/local/atmail/users 192.168.0.3(rw,no_root_squash)

This adds a permission entry for 192.168.0.3 (server2.local) to access your NFS share. Afterwards, restart NFS:

service nfs restart
service nfslock restart

Login afterwards to server2.local, then restart NFS services:

service nfs restart
service nfslock restart

Try to mount the mount point from server1.local afterwards, you will need to create the /usr/local/atmail/users directory if it does not already exist:

mount -t nfs 192.168.0.2:/usr/local/atmail/users /usr/local/atmail/users

Now the directory is shared between the servers.

Installing The Second Instance Of atmail

Now you can install the second atmail instance. The steps are the same as the first atmail instance.

Mail Server Installation

Using Rsync to synchronize the atmail application between frontends

After installing atmail on the second instance, you are required to setup a cron job to rsync the atmail application directory between servers. This ensures any updates to the atmail application on the master frontend, are replicated between the other server instances.

On each frontend for atmail append the following script to /root/rsync-atmail.sh

#!/bin/sh
 
/usr/bin/rsync -e ssh -avr --delete --exclude "users/" --exclude "tmp/" --exclude "webmail/log" --exclude "mailserver/spool/" --exclude "*.pid" root@192.168.0.2:/usr/local/atmail /usr/local

After applying the script, set execute permissions:

chmod 755 /root/rsync-atmail.sh

Next, edit the /etc/crontab and define an rsync push every 10 minutes to synchronize the application directory folder.

# Rsync for atmail
0,10,20,30,40,50 * * * * root /root/rsync-atmail.sh

Once setup

Configuring the Dovecot IMAP server

Note: These configuration parameters are only necessary for multiservers setups that use NFS. If using GlusterFS or Unison, skip this step.

Dovecot needs additional configuration parameters before it can work with NFS. You will need to add this at the top of your /usr/local/atmail/mailserver/etc/dovecot.conf file:

mmap_disable = yes
dotlock_use_excl = yes
mail_nfs_storage = yes
mail_nfs_index = no

Add this to both files on both servers. Restart atmail afterwards on both machines:

/etc/init.d/atmailserver restart

DNS Round Robin

Load Balancing

Load balancing can take many forms. Some may require specialized load balancing hardware to split the load between servers, while some may prefer sharing the load via DNS tweaks. This portion of the guide will guide you through setting up a DNS round robin.

The DNS round robin works on the principle that both machines have equal priority. This means that when a user sends a request for Web, SMTP, or POP/IMAP service, a server is randomly picked to handle the load.

For this guide, we will have three machines: server1.domain, server2.domain and server3.domain.

First, each machine must be defined in your zone file:

server1.domain.com.         IN      A       55.55.55.151
server2.domain.com.         IN      A       55.55.55.152
server3.domain.com.         IN      A       55.55.55.153

The hostname for each domain is placed on the left side. The IP addresses are in the right side. Set your hostname to your preferences.

DNS MX records can now be assigned. You will need to assign the MX records equal priority:

domain.com.           IN      MX      10       server1.domain.com
domain.com.           IN      MX      10       server2.domain.com
domain.com.           IN      MX      10       server3.domain.com

Replace "domain.com" with your domain-name. Next, you will need to add a subdomain for all POP3, IMAP, or Web requests to point to. For this example, we will use "mail":

mail  IN      A       55.55.55.151
mail  IN      A       55.55.55.152
mail  IN      A       55.55.55.153

You can now restart your nameserver. This will then apply the changes.

To test, type this command:

dig @nameserver domain.com

Replace "nameserver" with your nameserver's hostname or IP. Replace "domain.com" with your domain name. This should show something similar to:

;; QUESTION SECTION:
;domain.com.                    IN      MX
  
;; ANSWER SECTION:
domain.com.             86400   IN      MX      0 server3.domain.com.
domain.com.             86400   IN      MX      0 server1.domain.com.
domain.com.             86400   IN      MX      0 server2.domain.com.

Make sure that the "ANSWER SECTION" corresponds to your MX records. Next, we will test the domain name settings for your subdomain. It can be done via this command:

nslookup mail.domain.com

Replace "domain.com" with your domain. It will show something similar to:

nslookup mail.domain.com
 
Server:         127.0.0.1
Address:        127.0.0.1#53
  
Name:   mail.domain.com
Address: 55.55.55.151
Name:   mail.domain.com
Address: 55.55.55.152
Name:   mail.domain.com
Address: 55.55.55.153

Once verified, you can then set the servers to accept connections.

Have more questions? Submit a request

Comments


Contact our support team


+61 (7) 5357 6605       support@atmail.com