pre-commit hooks and terraform- a safety net for your repositories

tools

I’m the only infrastructure person on a number of my projects and it’s sometimes difficult to find someone to review pull requests. So, in self-defence, I’ve adopted git precommit hooks as a way to ensure I don’t make certain tedious mistakes before burning through peoples time and goodwill. In this post we’ll look at how pre-commit and terraform can be combined.

pre-commit is “A framework for managing and maintaining multi-language pre-commit hooks” that has a comprehensive selection of community written extensions. The extension at the core of this post will be pre-commit-terraform, which provides all the basic functionality you’ll need.

Before we start you’ll need to install pre-commit itself. You can do this via your package manager of choice. I like to run all my python code inside a virtualenv to help keep the versions isolated.

$ pip install pre-commit --upgrade
Successfully installed pre-commit-1.10.4

To keep the examples realistic I’m going to add the precommit hook to my Terraform SNS topic module. Mostly because I need it on a new project; and I want to resolve the issue raised against it.

# repo cloning preamble
git clone git@github.com:deanwilson/tf_sns_email.git
cd tf_sns_email/
git co -b add_precommi

With all the preamble done we’ll start with the simplest thing possible and build from there. First we add the basic .pre-commit-config.yaml file to the root of our repository and enable the terraform fmt hook. This hook will ensure all our terraform code matches what would be produced by running terraform fmt over your codebase.

cat <<EOF > .pre-commit-config.yaml
- repo: git://github.com/antonbabenko/pre-commit-terraform
  rev: v1.7.3
  hooks:
    - id: terraform_fmt
EOF

We then install the pre-commit within this repo so it can start to provide our safety net.

$ pre-commit install
pre-commit installed at /tmp/tf_sns_email/.git/hooks/pre-commit

Let the pain commence! We can now run pre-commit over the repository and see what’s wrong.

$ pre-commit run --all-files
[INFO] Initializing environment for git://github.com/antonbabenko/pre-commit-terraform.
Terraform fmt............................................................Failed
hookid: terraform_fmt

Files were modified by this hook. Additional output:

main.tf
outputs.tf
variables.tf

So, what’s wrong? Only everything. A quick git diff shows that it’s not actually terrible. My indentation doesn’t match that expected by terraform fmt so we accept the changes and commit them in. It’s also worth adding .pre-commit-config.yaml in too to ensure anyone else working on this branch gets the same precommit checks. Once the config file is commited you should never again be able to commit incorrectly formatted code as the precommit will prevent it from getting that far.

A second run of the hook and we’re back in a good state.

$ pre-commit run --all-files
Terraform fmt..............Passed

The first base is covered, so let’s get a little more daring and ensure our terraform is valid as well as nicely formatted. This functionality is only a single line of code away as the pre-commit extension does all of the work for us:

cat <<EOF >> .pre-commit-config.yaml
    - id: terraform_validate_with_variables
EOF

This line of config enables another of the hooks. This one ensures all terraform files are valid and that all variables are set. If you have more of a module than a project and are not supplying all the possible variables you can change terraform_validate_with_variables to terraform_validate_no_variables and it will be much more lenient.

New config in place we rerun the hooks and prepare to be disappointed.

> pre-commit run --all-files
Terraform fmt..................................Passed
Terraform validate with variables..............Failed
hookid: terraform_validate_with_variables


Error: 2 error(s) occurred:

* provider.template: no suitable version installed
  version requirements: "(any version)"
  versions installed: none
* provider.aws: no suitable version installed
  version requirements: "(any version)"
  versions installed: none

And that shows how long it’s been since I’ve used this module; it predates the provider extraction work. Fixing these issues requires adding the providers, a new variable (aws_region) to allow specification of the AWS region, and adding some defaults. Once we fix those issues the precommit hook will fail due to the providers being absent, but that’s an easy one to resolve.

...
* provider.template: no suitable version installed
  version requirements: "1.0.0"
  versions installed: none
...

> terraform init

Initializing provider plugins...
- Checking for available provider plugins on https://releases.hashicorp.com...
- Downloading plugin for provider "template" (1.0.0)...
- Downloading plugin for provider "aws" (1.30.0)...

One more precommit run and we’re in a solid starting state.

Terraform fmt.............................Passed
Terraform validate without variables......Passed

With all the basics covered we can go a little further and mixin the magic of terraform-docs too. By adding another line to the pre-commit config -

cat <<EOF >> .pre-commit-config.yaml
    - id: terraform_docs
EOF

And adding a placeholder anywhere in the README.md -

+### Module inputs and outputs
+
+<!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
+<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
+
+

terraform-docs will be invoked and add generated documentation to the README for all of the variables and outputs. If they ever change you’ll need to review and commit the differences but the hooks will stop you from ever going out of sync. Now we have this happening automatically I can remove the manually added, and error prone, documentation for variables and outputs. And be shamed into adding some useful descriptions.

pre-commit hooks will never replace a competent pull request reviewer but they help ensure basics mistakes are never made and allow your peers to focus on the important parts of the code, like structure and intent, rather than formatting and documentation consistencies. All of the code changes made in this post can be seen in the Add precommit pull request