Posts

Server setup ISPConfig 3 with Ubuntu 12.04

Note: This is work in progress! You should probably test this in a virtual machine first and play around with various settings.

There are a lot of tutorials over the Internet about how to configure an Ubuntu server for ISPConfig. I usually follow them, after all there aren’t so many variations on how to setup a LAMP. But I always find some things that bother me. I don’t claim my setup is “perfect”, but please read and make your own decisions.

Installing Ubuntu 12.04

Disk Partitioning

First of all, do yourself a favor and use at least 2 HDDs for the server you are going to setup. It doesn’t matter if they are of different sizes as long you have enough space on the smallest one. If they have same size, the better. The reason why I am insisting on using at least 2 HDDs is because you will be able to setup software RAID1. There are pro and cons of using software RAID1, but I think it’s better to have it. It’s not even hard to configure RAID1:

  • create 2 equal size partitions, one on each drive
  • mark them as “Use as: physical volume for RAID”
  • go to “Configure software RAID” and “Create MD device” -> RAID1, 2 disks, 0 spare and select the two partitions you have marked for RAID (probably /dev/sda1 and /dev/sdb1)

You should see now RAID1 device #0 which is commonly known as /dev/md0. Congrats, you have just finished setting up software RAID1! Press Alt+F2, Enter and check out the progress of RAID sync:

cat /proc/mdstat

Note: If you are using a system that has (U)EFI you might want to partition your disks before using GParted and you will need to create an EFIBoot partition of about 100MB at the beginning of HDD. To keep things looking nice you can create 2x 128MB partitions on both harddisks(one on each). More details about this on my previous article “Install Ubuntu 12.04 with RAID/LVM on UEFI system”

I prefer to use LVM over the RAID1 I’ve just created because I find it easier to manage this way. I usually create different logical volumes/partitions for:

  • /boot: this one is usually 128MB, for kernel; you skip it if you want
  • /: make this one about 10-20GB, for system
  • /tmp: can be anywhere from 2GB to 10GB, set nosuid and noexec for it
  • /var: this is where there are web files and databases, so I usually use the rest of the space for it; also set usrquota and grpquota for this

If you are using UEFI:You will need 1 vfat32 partition for EFIBoot and 1 ext4 partition for /boot, outside the software RAID. After playing around with all kinds of setups, this was the only working solution I have found. The bad thing with this setup is that in case of disk failure you will have to recover either EFI partition or/and the /boot partition. Sorry, it sucks, but I didn’t find a better way to do it.

Here is how to setup LVM

You can use LVM over RAID1 or simply use LVM. Here are the steps:

  • “Configure the Logical Volume Manager” and use /dev/md0 as Physical Volume.
  • Create a Volume Group and named it “VolGroup00” (or whatever you want)
  • Create as many Logical Volumes as you need, set the right size for them (you can resize later). I usually name them LogVolRoot (for /) or LogVolTmp (for /tmp)
  • When you are done click “Finish”

You now have successfully setup LVM over RAID1. Just one more step and you are done partitioning. For each of the LVMs you have created, click on them and set the following:

  • Use as: file system of your choice; I use XFS because of fs freeze and resize on the fly, feel free to use ext4 or whatever you like most in the end I used ext4 because of quota issues with ISPConfig 3
  • Format the partition: yes
  • Mount point: put here where you want to mount it (be it /, /boot, /tmp, /var, /srv)
  • Mount options: set nosuid and noexec for /tmp and usrquota and grpquota for /var if you have them

Note: About swap partition(s) … there are many approaches to this. You can have 2 swap partitions, one on each drive, set same priority for both and put them outside RAID/LVM. Or you can create a LV for swap. If the harddisks don’t have the same size you can create the swap on the free space left. Recommended swap size is (according to RHEL) 2* RAM if you have less than 2GB, and 2+ RAM if you have over 2GB. I had 2 HDDs, one of 500GB and one of 250GB so I created the swap on the free space left on the big drive. The rest of the space I’ve created a partition, put XFS on it and mounted in /srv for later use (probably some FTP storage, whatever is not important)

Installing software and ISPConfig 3

For the most part I’ve followed this excellent tutorial The Perfect Server – Ubuntu 12.04 LTS (Apache2, BIND, Dovecot, ISPConfig 3), but there are some minor things changed.

First of all I don’t like to disable AppArmor as suggested in the tutorial. From my experience is not that hard to figure out when you screwed up things. Most of the times you will have problems with AppArmor if you change data_dir for MySQL or you made changes to Apache, etc. If you have recently made changes to configuration of MySQL(let’s say you have moved database files from /var/lib to /srv/mysql) and the server is not starting up take a quick look at /var/log/syslog and look up for apparmor entries. If you see something like denied, edit the file /etc/apparmor.d/usr.sbin.mysqld and try to find the entries related to the directories modified. For /var/lib you will find 2 entries similar to these:

  /var/lib/mysql/ r,
  /var/lib/mysql/** rwk,

Either replace /var/lib with /srv/mysql or append 2 new lines, your choice. Restart AppArmor and this should be all. MySQL should start now.

Also in the tutorial at some point you are advised to comment out this line in /etc/mysql/my.cnf:

bind-address           = 127.0.0.1

You only need to do this if you plan to access the MySQL server/databases from another server(you have the website files on another server, you need to set replication). But if all you want to do is to run web sites that use “host= localhost” for their database settings don’t comment out that line. Most of the times you should be fine with MySQL listening only on local interface.

Install MySQL

apt-get install mysql-client mysql-server
mysql_secure_installation

I know there is a lot of debate about using innodb_file_per_table, but after getting a corrupted ibdata1 file and losing all the InnoDB tables I will go with having 1 file per table.

Install Apache2 + PHP-FPM

apt-get install apache2 apache2.2-common apache2-doc apache2-mpm-prefork apache2-utils libexpat1 ssl-cert libapache2-mod-php5 php5 php5-common php5-gd php5-mysql php5-imap phpmyadmin php5-cli php5-cgi libapache2-mod-fcgid apache2-suexec php-pear php-auth php5-mcrypt mcrypt php5-imagick imagemagick libapache2-mod-suphp libruby libapache2-mod-ruby

Also I found I was missing php5-curl package:

apt-get install php5-curl

Enable required mods:

a2enmod suexec rewrite ssl actions include

If you plan to use PHP-FPM and fastcgi mod (available in ISPConfig 3.0.5 or svn release):

apt-get install libapache2-mod-fastcgi php5-fpm
a2enmod fastcgi

Install Postfix and Courier

I am only installing Postfix without Amavis and AV/AS capabilities since I am planing to integrate ISPConfig with Zimbra in the future. I find SquirrelMail to be dated and I prefer the modern look and all the features available in the Zimbra Open Source edition. More on the integration of ISPConfig and Zimbra in a future article. Back to work:

apt-get install postfix postfix-mysql postfix-doc openssl getmail4
apt-get install courier-authdaemon courier-authlib-mysql courier-pop courier-pop-ssl courier-imap courier-imap-ssl libsasl2-2 libsasl2-modules libsasl2-modules-sql sasl2-bin libpam-mysql openssl courier-maildrop

Install PureFTPD

apt-get install pure-ftpd-common pure-ftpd-mysql

Install BIND (named, DNS)

apt-get install bind9 dnsutils

Install fail2ban and rkhunter

apt-get install fail2ban rkhunter binutils

Vlogger, Webalizer, Awstats

If you don’t use Awstats you can comment out all the entries in /etc/crond.d/awstats.

apt-get install vlogger webalizer awstats geoip-database libclass-dbi-mysql-perl

Quota tools

apt-get install quota quotatool

Install ISPConfig 3

cd /tmp
wget http://www.ispconfig.org/downloads/ISPConfig-3-stable.tar.gz
tar xfz ISPConfig-3-stable.tar.gz
cd ispconfig3_install/install/
php -q install.php

Fix /etc/postfix/master.cf:

sed -i -e 's/smtpd_bind_address/smtp_bind_address/' /etc/postfix/master.cf

In case you have missed the note at the top of the article: This is Work in Progress!

Apache2 worker vs prefork for ISPConfig benchmark

I’ve been running ISPConfig latest version(3.0.4) on Amazon cloud t1.micro instance for some time to host several small sites, mostly WordPress. I’m quite happy with the performance of the instance. The OS is Ubuntu 10.04 LTS. Until recently I’ve used the default mpm which is prefork, but I decided to test out worker also. If you are wondering I use mod_fcgid for all the sites. That being said I performed several tests with ab (apache benchmark) to see which mpm can server most requests per second.

While I do not claim this is the best setup, I think worker is better suited for me. Some people said they had problems because of mpm worker. So far so good, but will update this post if there are any issues.

Test results:

preforkworker
Concurrency Level: 32
Time taken for tests: 7.834 seconds
Complete requests: 5000
Failed requests: 0
Write errors: 0
Keep-Alive requests: 4972
Total transferred: 84831033 bytes
HTML transferred: 83206915 bytes
Requests per second: 638.27 [#/sec] (mean)
Time per request: 50.136 [ms] (mean)
Time per request: 1.567 [ms] (mean, across all concurrent requests)
Transfer rate: 10575.21 [Kbytes/sec] received
Concurrency Level: 32
Time taken for tests: 7.096 seconds
Complete requests: 5000
Failed requests: 0
Write errors: 0
Keep-Alive requests: 4968
Total transferred: 84877824 bytes
HTML transferred: 83247322 bytes
Requests per second: 704.63 [#/sec] (mean)
Time per request: 45.414 [ms] (mean)
Time per request: 1.419 [ms] (mean, across all concurrent requests)
Transfer rate: 11681.17 [Kbytes/sec] received

Creating consistent backups for EBS with EXT4 and quota

What’s this about?
Data security and backups are very important aspects when you work with servers, especially if you are using cloud infrastructure. I am using AWS(Amazon Web Services) as my preferred IaaS, so the following how-to is tailored for Amazon EC2 instances using EBS as storage for the web sites files. On my instance I have Ubuntu 10.04 LTS installed and on top of it I run ISPConfig 3.0.4(latest version at the moment I write this article). Some of the programs required to run this setup were already installed, but it should be pretty obvious if you miss anything. If you need help you can either leave a comment or contact me via email.

The following setup will allow you to create an EBS using EXT4 as file system, with quota enabled on it(for ISPConfig) and weekly backups of the EBS. In case of instance failure you should be able to launch a new instance and attach the EBS, without losing any web sites files. In case of EBS failure you can recreate one from the most recent snapshot.

Create an EBS in the same zone as your instance and attach it to your instance as /dev/sdf. This can be easily done from AWS Management Console.

Install xfsprogs

sudo apt-get install xfsprogs

Create EXT4 filesystem on /dev/sdf

sudo mkfs.ext4 /dev/sdf

Now mount it temporarily

sudo mkdir /mnt/ebs
sudo mount /dev/sdf /mnt/ebs

Stop the apache2 web server and copy the files to /mnt/ebs

sudo service apache2 stop
cd /mnt/ebs
sudo cp -rp /var/www/* .

Prepare quota

touch quota.user quota.group
sudo chmod 600 quota.*

Add the entry to /etc/fstab

/dev/sdf /var/www ext4 noatime,nobootwait,usrjquota=quota.user,grpjquota=quota.group,jqfmt=vfsv0 0 0

Unmount the EBS and remount it to /var/www

sudo umount /dev/sdf
sudo mount /dev/sdf /var/www -o noatime,usrjquota=quota.user,grpjquota=quota.group,jqfmt=vfsv0

Enable quota

sudo quotacheck -avugm
sudo quotaon -avug

Start the apache2 web server and check that the web sites are working properly

sudo service apache2 start

Install ec2-consistent-snapshot script for weekly backups of EBS

sudo add-apt-repository ppa:alestic
sudo apt-get update
sudo apt-get install -y ec2-consistent-snapshot

Prepare first snapshot(I assume the cron will run as root user, hence I create the awssecret file in /root directory)

sudo touch /root/.awssecret
sudo chmod 600 /root/.awssecret

Edit .awssecret and add following lines, in this order, replacing ACCESS_KEY_ID and SECRET_ACCESS_KEY with your own, both can be found under Account->Security Credentials:

ACCESS_KEY_ID
SECRET_ACCESS_KEY

Test the snapshot creation with debug mode activated, replace VOLUME_ID with the right volume ID:

sudo ec2-consistent-snapshot --debug --description "snapshot $(date +%Y-%m-%d-%H:%M:%S)" --freeze-filesystem /var/www vol-VOLUME_ID

If everything went well you should be able to see your new snapshot in the AWS Management Console.

Finally add this to your root crontab (by running sudo crontab -e):

@weekly /usr/bin/ec2-consistent-snapshot --debug --description "snapshot $(date +'%Y-%m-%d %H:%M:%S')" --freeze-filesystem /var/www vol-VOLUME_ID>>/var/log/backup.log 2>&1

Make sure you put the correct VOLUME_ID!

This should be all, you now have all your web sites on EBS, quota is enabled and weekly backups enabled. I think I pretty much nailed everything you need in order to perform this setup, but if there are any issues feel free to leave a comment. Also I love getting feedback so if you found this article useful leave a comment also 🙂

ISPConfig3 running on Ubuntu 9.10 on AWS

This is an idea on how to set ISPConfig 3 on Amazon EC2 for web hosting. It’s still work in progress and I await for any constructive feedback. Since English is not my first language there might be some grammatical errors.

Prerequisites:

  • AWS Account
  • Some Linux experience

Part I – Setting up AWS

I have done all the setup of instance and ebs from the Amazon Management Console. It has a very intuitive interface and it’s so easy to work with.

Optional: You can purchase reservation for an instance if you are planing to run it 24/7. To do so go to Instances -> Reserved Instances and click on Purchase Reserved Instance. For example a c1.medium instance reservation for 1 year costs 455$ one time fee and the hourly cost of the instance goes to 0.06$/hour. It’s your decision, do the math if it’s worth it or not to go with reserved instances.

  1. From the Management Console go to Volumes and Create Volume. I created a 40 GB volume in us-east-1a zone, choose an appropriate value for your needs.
  2. Next step which is also optional is to get an Elastic IP for your instance. Go to Elastic IPs and click Allocate New Address.
  3. You will need a keypair to access your instance. You can create one before launching the instance or create one when you also setup the new instance. Your choice. I created one before by going to Key Pairs -> Create Key Pair.

Part II – Launching the instance

Instance type used was c1.medium, the AMI for it was Ubuntu 9.10 Karmic 32bit ami-1515f67c from Canonical Images for Amazon EC2.

Go to Instances and press Launch Instance and make sure you set it to use ami-1515f67c. If you need to use a different zone or a 64 bit instance you can check the alestic page which is a wonderful resource for Ubuntu images for EC2. I would also suggest to join their EC2 Ubuntu Google Grup if you are serious about AWS + Ubuntu.

After the instance is up and running attach the Elastic IP and the Volume you have created to it. To attach the IP go to Elastic IPs and select Associate, for the EBS go to Volumes. I attached the EBS as /dev/sdb to the instance.

Later Edit: You will need to also modify the Security Groups and permit access on ports 22(SSH), 80(HTTP), 8080(ISPConfig 3), 20-21(FTP) -> still need more work to support PASV.

Part III – Install ISPConfig 3

Before proceeding with ISPConfig 3 setup connect to your instance and update the packages.

sudo su
apt-get update
apt-get upgrade

Note: You will have to connect to your instance using the ubuntu username and the key pair you generated for that instance. It won’t allow you to connect directly as root.

To install the ISPConfig 3 I followed the HowtoForge excellent tutorial with a small exception: I skipped the part of setting up the quota since I was planning to use the EBS as storage for sites and MySQL databases. The rest of the tutorial was just perfect.

Part IV – Setting up the EBS and ISPConfig to work with it

Most of the stuff and talk about EBS and MySQL snapshots is discussed on Amazon developer forums in the thread called Tutorial: Running MySQL on Amazon EC2 with EBS (Elastic Block Store). Make sure you read the discussion before or after you are done with this post.

apt-get install dmsetup
modprobe dm_mod
modprobe dm_mirror
modprobe dm_snapshot
mkdir /dev/.static/dev/mapper -pv

Create ext3 filesystem:

mkfs.ext3 /dev/sdb

Setup it with dmsetup

echo 0 `blockdev --getsize /dev/sdb` linear /dev/sdb 0 | dmsetup create ebs

Add the records to fstab

echo "/dev/mapper/ebs /ebs ext3 noatime,usrjquota=aquota.user,grpjquota=aquota.group,jqfmt=vfsv0 0 0" >> /etc/fstab

Setup quota

touch /ebs/aquota.user /ebs/aquota.group
chmod 600 /ebs/aquota.*
mount -o remount /ebs

Enable quota on ebs

quotacheck -avugm -F vfsv0 /ebs
quotaon -avug /ebs

For websites:

mkdir /ebs/sites
ln -s /ebs/sites /var/www/clients

Jailkit:

Set Jailkit chroot home: /ebs/home/[username]

mkdir /ebs/home

MySQL:

/etc/init.d/mysql stop
mkdir /ebs/mysql/data /ebs/mysql/log -pv
mv /var/lib/mysql /ebs/mysql/data
ln -s /ebs/mysql/data/mysql /var/lib/mysql
mv /var/log/mysql /ebs/mysql/log
ln -s /ebs/mysql/log/mysql /var/log/mysql

To make a snapshot of the database, go to the mysql console (msqyl -p)

FLUSH TABLES WITH READ LOCK;
SHOW MASTER STATUS;
SYSTEM dmsetup suspend /dev/mapper/ebs

Take your snapshot of the EBS

SYSTEM dmsetup resume /dev/mapper/ebs
UNLOCK TABLES;

Note:This is still work in progress, any advices on how to improve it are greatly appreciated and any constructive criticism also.

ISPConfig3: Updating to myDNS-ng

If you have followed the tutorials “the perfect setup” for installing ISPConfig 3 chances are that you have an older myDNS version (1.1.0 or so). Since I had a few problems with that version I decided to update to myDNS-ng, the latest version.

In this article I will refer to rpmbuilder user, a user I create to compile and/or make RPMs. For more information on how to set it up check this link.

Also the instructions are for CentOs 5.4(but will probably work for any other 5.x version).

su - rpmbuilder
cd /tmp
wget http://downloads.sourceforge.net/project/mydns-ng/mydns/1.2.8.27/mydns-1.2.8.27.tar.gz?use_mirror=garr
tar zxvf mydns-1.2.8.27.tar.gz
cd mydns-1.2.8
./configure
make

Everything should be ok untill now. I had no problems to compile it. Then I decided to use checkinstall.

sudo yum install checkinstall
checkinstall

It will ask you a few questions, answer them and you should have a rpm package in /usr/src/redhat/RPMS. It will tell you anyway where he created the rpm.
After that I removed the old mydns packages and installed the new one as root:

service mydns stop
yum remove mydns mydns-mysql
yum install /usr/src/redhat/RPMS/i386/mydns-1.2.8-1.i386.rpm --nogpgcheck

After that I needed to create mydns start script:

vi /etc/init.d/mydns
chmod +x /etc/init.d/mydns

I used the script posted by rukus77 on howtoforge. Make sure you read all the comments on that forum (especially the one containing [ -f /usr/sbin/mydns ] || exit 1).
The last step is to run ISPConfig 3 setup:

cd ispconfig3_install/
cd install
php -q update.php

When asked if to reconfigure services answer yes, the rest you can answer no.

That would be all, thanks for reading.