Use AWS CLI to automate the removal and addition of instances in your ELB

Hello!

Sometimes its necessary to automate the removal and addition of instances in your elastic load balancer. Perhaps for the purposes of auto scaling or deploying updates to your web application. Either way, there is many tools at the disposal of the systems administrator to automate this process. Below we will share some simple steps as well as some (very) simple scripts to make it that much easier to manipulate the instances that are receiving live traffic via the ELB.

Install AWS CLI

This is pretty straightforward (and obvious). Amazon themselves provide a great guide to installing the AWS Cli toolset on your linux distribution. Below I’ll provide the shorthand for setting up AWS Cli on CentOS/RedHat or Ubuntu/Debian systems.

CentOS/RedHat

Ubuntu/Debian

So simple, right? There are other ways to install the toolset, such as through Python or to download the source and install directly from the source. After installing it, you will want to configure it with the access credentials in order to authenticate against your AWS account :

Before you do that, it might be a good idea to create a new IAM user with restricted access.

Create IAM user in AWS Security Console to access only your ELB

Restricting access for your IAM user is a good best practice. It will ensure that the access you delegate will never go beyond what was originally intended and will also mitigate any damage a malicious user might do should they be able to gain access to the credentials.

What you would want to do is create a group first, with the following two policies attached : AmazonEC2ReadOnlyAccess (a pre-made policy that you can search for and attach automatically), and a custom policy detailed below.

The policy above will allow the users in the IAM group to only access the specified ELB (where “your-elb-name” is specified). If you are in a different availability zone, you would also want to change us-east-1 to whatever zone your in.

Once the policy is attached to the group, then you simply need to create the user, add them to the group you created and create the access credential key/secret to use with the aws configure command.

The purposes of the script for us was to create a script on each actual instance so that we could simply run the script locally and automatically take the instance out of the pool. This means, for us, we ran the aws configure command on each instance that the script was running on. If you are using a centralized server (i.e. Jenkins, Ansible, Puppet, etc) then your script may look different. Perhaps your script in this case would parse the instances that are currently active in the ELB, and then iterate through each, taking them out of the ELB, running the update (or whatever you need to do) and then putting them back before moving on to the next.

Bash script to automate adding and removing servers to an ELB

This bash script is dead simple. We are simply grabbing the instance id of the script its running on and then read the user input to determine if the request is to add or remove the instance in question from the ELB. The script can definitely be improved further to perhaps include an error checker to make sure if you are removing the instance that there is at least 1 other active instance in the ELB before doing that (to avoid outages).

Pretty straightforward! Again, better error checking and perhaps parsing the output of the aws commands may add better checks and balances to this kind of manipulation of your ELBs. For that level of checking and parsing it might be worth exploring using Python instead of Bash.

Automated Amazon EBS snapshot backup script with 7 day retention

Hello there!

We have recently been implementing several different backup strategies for properties that reside on the Amazon cloud platform.

These strategies include scripts that incorporate s3sync and s3fs for offsite or redundant “limitless” backup storage capabilities. One of the more recent strategies we have implemented for several clients is an automated Amazon EBS volume snapshot script that only keeps 7 day retention on all snapshot backups.

The script itself is fairly straightforward, but took several dry-runs in order to fine tune it so that it would reliably create the snapshots, but more importantly would clear out old snapshots older than 7 days.

You can see the for loop for deleting older snapshots. This is done by parsing snapshot dates, converting the dates to a pure numeric value and comparing said numeric value to a “7 days ago” date variable.

Take a look at the script below, hopefully it will be useful to you! There could be more error checking, but that should be fairly easy to do.

Massive Amazon Route53 API Bind Zone Import Script

Hello there,

Occasionally some of our managed services work has us dealing directly with other cloud providers such as Amazon. One of our clients set a requirement to migrate over 5,000 domain’s to Amazon’s Route53 DNS service.

There was little doubt that this could be automated, but since we have never done this massive of a deployment through Amazon’s API directly, we thought it might be interesting to post the process as well as the script through which we managed the import process.

Essentially the script utilizes a master domain name list file as its basis for looping through the import. The master list refers to the bind zone files and imports them into Amazon’s Route53 via the Cli53 tool package.

One final note, the script outputs all completed domain imports into a CSV file with the following format :

This is because when facilitating the actual nameserver change request, all the nameservers assigned to domains when imported to Route53 are randomly generated, so the script has to keep track of these nameserver/domain associations.

The script isn’t perfect and could benefit from some optimizations and more error checking (it does a lot of error checking already, however), but here it is in its entirety. We hope you will have some use for it!