Use Ansible to Expand CloudFormation Templates

After a previous comment about “ templating CloudFormation JSON from a tool higher up in your stack” I had a couple of queries about how I’m doing this. In this post I’ll show a small example that explains the work flow. We’re going to create a small CloudFormation template, with a single Jinja2 embedded directive, and call it from an example playbook.

This template creates an S3 bucket resource and dynamically sets the “DeletionPolicy” attribute based on a value in the playbook. We use a file extension of ‘.json.j2’ to distinguish our pre-expanded templates from those that need no extra work. The line of interest in the template itself is "DeletionPolicy": "{{ deletion_policy }}". This is a Jinja2 directive that Ansible will interpolate and replace with a literal value from the playbook, helping us move past a CloudFormation Annoyance, Deletion Policy as a Parameter. Note that this template has no parameters, we’re doing the work in Ansible itself.

    $ cat templates/deletion-policy.json.j2 

    {
      "AWSTemplateFormatVersion": "2010-09-09",

      "Description": "Retain on delete jinja2 template",

      "Resources": {

        "TestBucket": {
          "DeletionPolicy": "{{ deletion_policy }}",
          "Type": "AWS::S3::Bucket",
          "Properties": {
            "BucketName": "my-test-bucket-of-54321-semi-random-naming"
          }
        }

      }
    }

Now we move on to the playbook. The important part of the preamble is the deletion_policy variable, where we set the value for later use in the template. We then move on the the 2 essential and one house keeping task.

    $ cat playbooks/deletion-policy.playbook 
    ---
    - hosts: localhost
      connection: local
      gather_facts: False 
      vars:
        template_dir: "../templates"
        deletion_policy: "Retain" # also takes "Delete" or "Snapshot"

Because the Ansible CloudFormation module doesn’t have an inbuilt option to process Jinja2 we create the stack in two stages. First we process the raw jinja JSON document and create an intermediate file. This will have the directives expanded. We then run the CloudFormation module using the newly generated file.

  tasks:
  - name: Expand the CloudFormation template for future use.
    local_action: template src={{ template_dir }}/deletion-policy.json.j2 dest={{ template_dir }}/deletion-policy.json

  - name: Create a simple stack
    cloudformation: >
      stack_name=deletion-policy
      state=present
      region="eu-west-1"
      template={{ template_dir }}/deletion-policy.json

The final task is an optional little bit of house keeping. We remove the file we generated earlier.

  - name: Clean up the local, generated, file
    file: name={{ template_dir }}/deletion-policy.json state=absent

We’ve only covered a simple example here but if you’re willing to commit to preprocessing your templates you can add a lot of flexibility, and heavily reduce the line count, using techniques like this. Creating multiple subnets in a VPC, adding route associations and such is another good place to introduce these techniques.