Working with multiple, related CloudFormation stacks can become quite taxing if you only use the native AWS command line tools. Commands start off gently - cfn-create-stack dwilson-megavpc-sns-emails --parameters "AutoScaleSNSTopic=testy@example.org" \ --template-file location/sns-email-topic.json but they quickly become painful. The two commands below each create stacks that depend on values from resources that have been defined in a previous stack. You can spot these values by their unfriendly appearance, such as ‘rtb-9n0tr34lac55’ and ‘subnet-e4n0tr34la’. Read on →

Once we started extracting applications into different logical CloudFormation stacks and physical templates, we began to notice quite a lot of duplication in our json when it came to declaring IAM rules. Some of our projects store their puppet, hiera and rpm files in restricted S3 buckets so allowing stacks access to them based upon environment, region, stack name and other criteria quickly becomes quite long-winded. After looking at a couple of dozen application templates and finding that over 30% of the json was IAM based it was time to find a different approach. Read on →

Structured facts in facter had become the Puppet communities version of ‘Duke Nukem Forever’, something that’s always been just around the next corner. Now that the facter 2.0.1 release candidate is out you can finally get your hands on an early version and do some experimentation. First we grab a version of facter 2 that supports structured facts from puppetlabs - # our play ground mkdir /tmp/facter && cd /tmp/facter # grab the code wget https://downloads. Read on →

One of the nice little conveniences I’ve started to use in my daily work with Amazon Webservices CloudFormation is the Guard::CloudFormation ruby gem. The Guard gem "is a command line tool to easily handle events on file system modifications" which, simply put, means “run a command when a file changes”. While I’ve used a number of different little tools to do this in the past, Guard presents a promising base to build more specific test executors on so I’ve started to integrate it in to more aspects of my work flow. Read on →

One of the biggest surprises of Config Management Camp 2014 for me was how interesting Canonicals orchestration management tool, Juju has become. Although I much preferred the name ‘Ensemble’. I attended the Juju session in an attempt to keep myself out of the Puppet room and was pleasantly surprised at how much Juju had progressed since I last looked at it. Rather than being another config management solution it allows you to model your systems using “charms”, which can be implemented using anything from a bash script to a set of chef/puppet cookbooks/modules, and instead focuses on ensuring that they run across your fleet in a predictable way while enforcing dependencies, even over multiple tiers, no matter how many tools you choose to use underneath. Read on →

I’m still new to Ansible and while it’s been interesting seeing how people are starting to use the tool, picking up bits and pieces from different blog posts is a little too hit and miss for my learning needs. When I spotted Ansible Configuration Management (PacktPub) I decided to take the plunge and see if it could provide me with a more consistent introduction. And it did. This book makes an ideal first stop for anyone wanting to learn Ansible. Read on →

As the Ansible/AWS investigations continue I had the need to lookup outputs from existing CloudFormation stacks. I spent ten minutes reading through the existing lookup plugins and came up with the Ansible CloudFormation Lookup Plugin. I’m not sure this is going to be our final solution. Michael DeHaan suggested that moving to a fact plugin might be better in terms of cleaner usage and easier testing, so I’m at the least going to implement a trial version of that. Read on →

I picked up a copy of Learning AWS OpsWorks during the PacktPub holiday sale. It was cheap, short and covered a AWS product that I’ve never had need to dig in to and knew very little about. The book takes you through creating a basic stack, the layers inside it and deploying an application to managed instances. Its coverage is very high level and doesn’t really go beyond a cursory explanation of the services used. Read on →

Back in November 2013 Amazon added a much requested feature to CloudFormation, the ability to conditionally include resources or their properties in to a stack. As an example I’m currently using this as a small cost saving measure to ensure only my production RDS instances have PIOPs applied to them while being able to build each environment from a single template. CloudFormation Conditionals live in their own section of a CloudFormation template. Read on →

I changed jobs midway through 2013 and quite quickly discovered that I’d been a little too buried in my role and not been keeping up other parts of my technical interests. As a first step I decided to put a very basic Pragmatic Investment Plan in place. Mostly as a simple way to ensure I actually started to get involved in non-work things again. Firstly I set myself the task of recording which books I actually read. Read on →