Constructing a large, multiple application, virtual datacenter with CloudFormation can quickly lead to a sprawl of different stacks. The desire to split things sensibly, delegate control of separate tiers and loosely couple as many components as possible can lead to a large number of stacks, lots of which need values from stacks created earlier in the run order. While it’s possible to do this with the native AWS CloudFormation command line tools, or even some clever bash Read on →

I’ve been doing my usual quarterly sweep of the always too full bookshelves and hit the usual dilemma of what to keep, what to donate to charity and what to recycle. Among the technical books in this batch is the ‘Sendmail Cookbook’, something I’ve always kept as a good luck charm to ward off the evil of needing to work with mail servers with m4 based configuration languages. Sendmail is one of those projects that I’ve not kept up with over the years. Read on →

Working with multiple, related CloudFormation stacks can become quite taxing if you only use the native AWS command line tools. Commands start off gently - cfn-create-stack dwilson-megavpc-sns-emails --parameters "AutoScaleSNSTopic=testy@example.org" \ --template-file location/sns-email-topic.json but they quickly become painful. The two commands below each create stacks that depend on values from resources that have been defined in a previous stack. You can spot these values by their unfriendly appearance, such as ‘rtb-9n0tr34lac55’ and ‘subnet-e4n0tr34la’. Read on →

Once we started extracting applications into different logical CloudFormation stacks and physical templates, we began to notice quite a lot of duplication in our json when it came to declaring IAM rules. Some of our projects store their puppet, hiera and rpm files in restricted S3 buckets so allowing stacks access to them based upon environment, region, stack name and other criteria quickly becomes quite long-winded. After looking at a couple of dozen application templates and finding that over 30% of the json was IAM based it was time to find a different approach. Read on →

Structured facts in facter had become the Puppet communities version of ‘Duke Nukem Forever’, something that’s always been just around the next corner. Now that the facter 2.0.1 release candidate is out you can finally get your hands on an early version and do some experimentation. First we grab a version of facter 2 that supports structured facts from puppetlabs - # our play ground mkdir /tmp/facter && cd /tmp/facter # grab the code wget https://downloads. Read on →

One of the nice little conveniences I’ve started to use in my daily work with Amazon Webservices CloudFormation is the Guard::CloudFormation ruby gem. The Guard gem "is a command line tool to easily handle events on file system modifications" which, simply put, means “run a command when a file changes”. While I’ve used a number of different little tools to do this in the past, Guard presents a promising base to build more specific test executors on so I’ve started to integrate it in to more aspects of my work flow. Read on →

One of the biggest surprises of Config Management Camp 2014 for me was how interesting Canonicals orchestration management tool, Juju has become. Although I much preferred the name ‘Ensemble’. I attended the Juju session in an attempt to keep myself out of the Puppet room and was pleasantly surprised at how much Juju had progressed since I last looked at it. Rather than being another config management solution it allows you to model your systems using “charms”, which can be implemented using anything from a bash script to a set of chef/puppet cookbooks/modules, and instead focuses on ensuring that they run across your fleet in a predictable way while enforcing dependencies, even over multiple tiers, no matter how many tools you choose to use underneath. Read on →

I’m still new to Ansible and while it’s been interesting seeing how people are starting to use the tool, picking up bits and pieces from different blog posts is a little too hit and miss for my learning needs. When I spotted Ansible Configuration Management (PacktPub) I decided to take the plunge and see if it could provide me with a more consistent introduction. And it did. This book makes an ideal first stop for anyone wanting to learn Ansible. Read on →

As the Ansible/AWS investigations continue I had the need to lookup outputs from existing CloudFormation stacks. I spent ten minutes reading through the existing lookup plugins and came up with the Ansible CloudFormation Lookup Plugin. I’m not sure this is going to be our final solution. Michael DeHaan suggested that moving to a fact plugin might be better in terms of cleaner usage and easier testing, so I’m at the least going to implement a trial version of that. Read on →

I picked up a copy of Learning AWS OpsWorks during the PacktPub holiday sale. It was cheap, short and covered a AWS product that I’ve never had need to dig in to and knew very little about. The book takes you through creating a basic stack, the layers inside it and deploying an application to managed instances. Its coverage is very high level and doesn’t really go beyond a cursory explanation of the services used. Read on →