When I began working with the Heroku team, one of the first tools I had to learn was Stem. Originally developed by Peter van Hardenberg and based upon Judo, Stem was built to ease the the pain that comes about when building custom Amazon EC2 instances.
Assuming that you are like me in that you have both a short attention span and little time on your hands, we’ll keep things simple. In subsequent posts I’ll go into more detail and better practices, but for now we’ll focus on creating a simple ‘base’ AMI that we can fabricate to our heart’s content.
Once complete, we should be able to launch 100 of these suckers if we had the means to pay for them.
First, install stem:
$ sudo gem install stem
Then add your AWS credentials to your environment (I add mine to my ~/.profile):
export AWS_ACCESS_KEY_ID=XXXXXX
export AWS_SECRET_ACCESS_KEY=XXXXXX
Don’t forget to do this:
. ~/.profile
Next…
Run the following to setup the basic directory structure:
$ mkdir -p stems/bare stems/base
$ touch stems/bare/bare.json
$ touch stems/base/base.json
We’ll start by building out a ‘bare’ instance, which will just be a standard public AMI. Once we get this guy up and running, we’ll make a few changes and then build a new AMI out of it for future use.
Let’s take a look at our bare.json:
{
"ami" : "ami-4a0df923",
"instance_type" : "t1.micro",
"key_name" : "default"
}
There are three parameters here, all of which are fairly self-explanatory. ami refers to the AMI we want to launch, being an Ubuntu Lucid instance in this case. instance_type tells us what sort of system. In our case we’re going to use a t1.micro, which is the cheapest instance size available. No need to break the bank here. key_name defines which SSH key we’re going to use. This name is managed in the AWS Console, so substitute as you need to.
Now that all of this is understood, let us go ahead and launch our bare instance:
$ stem launch bare/bare.json
New instance ID: i-XXXXXX
That returned line, New instance ID: i-XXXXXX, tell us the instance id that we’ll work with. Now do this to get a list of running instances:
$ stem list
------------------------------------------
Instances
------------------------------------------
i-17e5d17b 50.16.66.78 running default ...
Great. We have a running system and an IP to connect to. Just do this to get going:
$ ssh ubuntu@50.16.66.78
You should get a shell. Now let us make a minor customization to our instance. You can obviously do something more elaborate, but this will serve as a demonstration:
$ echo FOO > BAR
Now we have a file named ‘FOO’ in the home directory of the ubuntu user. Good enough for me. Let’s build a custom AMI off of this.
First, halt the instance:
$ sudo halt
Run stem list a couple of times to ensure that the instance has entered the stopped state:
$ stem list
------------------------------------------
Instances
------------------------------------------
i-17e5d17b 50.16.66.78 stopped default ...
Capture it like so:
$ stem create base i-17e5d17b
After a few minutes (it could take longer depending on how AWS is behaving), you should be able to run stem list and see something like this:
$ stem list
------------------------------------------
Instances
------------------------------------------
i-17e5d17b 50.16.66.78 stopped default ...
------------------------------------------
AMIs
------------------------------------------
base ami-487d8e21
Woo! You now have a base AMI ready with your customizations to be deployed as often as you like. You could do this via the AWS Console… or you could just create a new Stem definition to make this possible. We’re going to go the Stem route.
Create stems/base/base.json with the following info:
{
"ami" : "ami-487d8e21",
"instance_type" : "t1.micro",
"key_name" : "default"
}
As you can see, the only difference between this definition and the bare one is the AMI.
You can now launch it like so:
$ stem launch stems/base/base.json
If you give it a second to boot and then SSH into the box, you’ll see that our BAR file is in place.
Pretty cool, eh? Once you are done playing around, make sure you go back and run stem destroy against each running instance so you don’t rack up unexpected charges.
In future articles I’ll talk more about better ways to bootstrap these instances, configuration management, and using the stem api instead of the command line tool.