Code & Config – sharing the smart bits

Setting up backup policy on for EC2 servers is a three step process. You will need to backup your latest setup on EC2 instance, your files on Elastic Block and most importantly the customer – applicaton data in your database.

Creating backup of EC2 instance

You can backup your EC2 instance white easily Amazon EC2 Tools which. The tools consist of useful scripts which are easy to use, you only need to pass the required parameters. The process consist of three steps again. Number one – create a backup of the instance into the tmp directory on the server itself. Number two – upload the backup into Amazon S3. Number three – Register your own AMI based on the S3 backup.

Number 1 – Create backup of your instance

You might need to create tmp directory in /mnt. You will also need to save your amazon pk and cert keys into a folder on your server. I have used root in my sample code.

sudo ec2-bundle-vol -d /mnt/tmp -k /root/pk-yourpkcodewillcomehere.pem --cert /root/cert-yourcekcodewillcomehere.pem -u yourAWSaccounnumber

Do not forget to replace ‘pk-yourpkcodewillcomehere.pem’ with your pem kode, ‘cert-yourcekcodewillcomehere.pem’ with your cert code and ‘yourAWSaccounnumber’ with your AWS account number.

Number 2 – Move your backup to S3 server

sudo ec2-upload-bundle -b yourS3backup name -m /mnt/tmp/image.manifest.xml -a privatecode -s secretcode

Do not forget to replace ‘yourS3backup’ with your S3 backup name, ‘privatecode’ & ‘secretcode’ with your actual keys.

Number 3 – Register your AMI

Login to your Amazon Management Console. Click on AMIs link in the side bar and then click the button ‘Register NEW AMI’ you will see popup with following text – http://s3.amazonaws.com:80/. You will need to supply your bucket name and name of the xml manifest. (bucketname//image.manifest.xml).

Creating backup of Elastic Volume

Open your Amazon Management Console and click on the Volumes linke in the side bar and then click the button ‘Create Snapshot’. That is all to it. You will need to specify the name for the snapshot.

Creating Database backup

You can setup a cron job to backup the database every hour. You will need to create a script which will be called by the cron.

Lets explain what the script does. It will dump the mysql database into a sql file. It will create a variable with a date and time so every backup file has a unique name. It will compress the sql file into tgz file. It will remove the sql afterwords as it will not be needed anymore.
It will use the s3-bash script to push the backup file into your Amazon S3 bucket. It will remove the compress file from your server as it will not bee needed anymore.

You will need to download the s3-bash.0.02 tools from Google repository. As you can see in the example bellow I have stored them in the /mnt/data/scripts/ directory.

#!/bin/sh
mysqldump -uroot -p'password' --opt 'your_database_name' > /mnt/data/mysql_backup/'your_databse_name'.sql
dtb=dtb-$(date +%Y%m%d-%H%M%S).tgz
cd /mnt/data/mysql_backup
tar -zcvf $dtb 'your_databse_name'.sql
rm 'your_databse_name'.sql
/mnt/data/scripts/s3-bash.0.02/s3-put -v -k'your_public_key' -s 'path_to_your_secret_key' -T /mnt/data/mysql_backup/$dtb /'your_bucket_name'/$dtb
rm $dtb

As you can see I am storing the backup file temporarily in /mnt/data/mysql_backup/ but you can chose any folder.
Make sure your replace the values in quotes with your own terms. Please note that you will need to create a file with your secret key and not place the secret key directly as a variable. I advice you not to use vi editor when creating this file as the file needs to be exactly 40 characters and vi editor will add hiden character so you end up with 41 characters and the script will not work. I have used Coda editor to create this file and it worked well.

Once you create this file you will need to setup crontab so the file will be executed every hour. You can edit your crontab using following command

crontab -e

You will need to add the following code

00 */1 * * * /mnt/data/scripts/'nameofyourscript'

Do not forget to replace ‘nameofyourscript’ with the actual name of your script and adjust the path if you have stored the script in different location, as you can see I have stored it in the scripts folder on my elastic block /mnt/data/scripts/. The crontab syntax means that the script will be executed every hour. In general it follows the order minute hour day month…etc. Search google for crontab to get more details.

Please not that in my example the database is not going to be huge so I can afford to dump the whole database every hour but you might need better solution such is incremental backup if your database is going to be huge.