When writing puppet modules sometimes you need to ensure that certain classes are only used within your module itself. For example a class that implements functionality based on the local operating system that should only be used by your public class. While reading though the new puppetlabs-mcollective modules source I came across a new pattern I’d not seen used in puppet before that achieves this in a very elegant way and i thought it was worth a second look. Read on →

We’re currently moving some of our early stage dev prototypes to a more automated environment and as part of this work I’m converting command line AWS resource creation to parameterised CloudFormation templates that we can use to either run multiple stacks side by side or recreate the entire stack from development to production. It’s been quite a frustrating afternoon due to some tool chain related yak shaving and some nuances in how CloudFormation works. Read on →

The most interesting links in my twitter feed this week have all been about VMWorld 2013 and the new technologies VMWare is making available in vSphere 5.5. As I don’t actually use VMWare at work (we’re using KVM and AWS at the moment) very little of it is immediately relevant to me - but then I spotted the ‘VMWare Certified Associate - Data Center Virtualization’ announcement. There is some combination of both being stuck inside waiting for family to arrive and pure envy about all the new cool stuff that tempted me in to watching the (free) VMWare Data Center Virtualization Fundamentals training course, reading the exam blue print and making an attempt at the exam. Read on →

Disclaimer: I read the early release version of this book. Some details may change between the version I read the final, published one. The first two chapters of Heroku: Up and Running are my favourite of the book. After the usual selling and explanation of virtual servers and the arrival of ‘The Cloud’ the authors cover the concept, culture and technical architecture of Heroku. The technical details of a closed company are never covered as deeply as you’d like but the material here is interesting, explains quite a lot of Herokus main technology and approaches and how th platform has evolved to enhance an opinionated work flow that seems to suit many developers. Read on →

It’s been quite a while since I’ve been heavily involved in large scale A/B testing frameworks and after reading Bandit Algorithms for Website Optimization I feel like I’ve been shown a whole new toolkit in under 90 pages. This slim book quickly explains the basic principles of A/B testing and then introduces both one of the major problems with it, sending traffic to an inefficient option, and presents three algorithms based on studies of the multi-armed bandit problem that will introduce you to new, faster, ways to determine which trials are worth investigation and further exploration. Read on →

I’ve recently had the need to create a handful of small file based providers in puppet and while trundling uphill against the ParsedFile provider I decided to have a look at how custom providers are written using the Puppet augeas providers. My sample provider is a simple one, it manages entries in /etc/shells and was quite quick to write. While augeasproviders assumes you know how Puppet types and providers are structured it does present a very standardised way of using the augeas library in order to manipulate your target. Read on →

AWS CloudFormation gives developers and systems administrators an easy way to create and manage a collection of related AWS resources, provisioning and updating them in an orderly and predictable fashion. -- AWS CloudFormation Homepage I’ve gone from never having used Amazon CloudFormation to building multi- tier, cross region, many availability zone deployments in a couple of months and while digging through official documentation, support requests, blog posts and sample templates I’ve put together what I’ve come to view as the ‘Four Stages of CloudFormation’. Read on →

Ever since they started the Velocity Conference in the US I’ve wanted to attend one and this year I finally made it to the Santa Clara Velocity (2013). I was lucky enough to attend a tutorial in each of the four slots: Monitoring and Observability - Theo Schlossnagle Bits on the Wire - Mark Nottingham Using Amazon Web Services for MySQL at Scale - Laine Campbell Managing PostgreSQL with Ansible in EC2 - Jay Edwards I attended Mark and Theos talks based on upon previous experience of them as presenters. Read on →

While Puppet may get all the glory, Facter, the hard working information gathering library that can, seldom gets much exciting new functionality. However with the release of Facter 1.7 Puppetlabs have standardised and included a couple of useful facter enhancements that make it easier than ever to add custom facts to your puppet runs. These two improvements come under the banner of ‘External Facts’. The first allows you to surface your own facts from a static file, either plain text key value pairs or a specific YAML / JSON format. Read on →

Over time parts of your puppet manifests will become unneeded. You might move a cronjob or a users in to a package or no longer need a service to be enabled after a given release. I’ve recently had this use case and had two options - either rely on comments in the Puppet code and write an out of band tool to scan the code base and present a report or add them to the puppet resources themselves. Read on →