Testing with Goss

I’m a big fan of serverspec but there are times the ruby tool chain behind it can be an annoyance and result in lots of baggage being installed. This isn’t a major problem on development machines, where many of the gems will already exist, but on production hosts the runtime dependencies can be comparatively heavy. To avoid this I’ve started looking at possible alternatives and one young, but promising, project is Goss

• the tool for ‘Quick and Easy server validation’.

Goss is written in Go and so avoids any library dependency issues; it’s lovely to have a testing tool with such a minimal footprint. In terms of writing tests serverspec has the benefits of a higher level DSL and the descriptiveness that brings to your spec files. Goss checks on the other hand are defined in json, which to me makes them both less visually appealing and quite simple to generate. Below is an example Redis test suite for each tools.

# a sample, Redis server testing, goss config
{
    "package": {
        "redis": {
            "installed": true
        }
    },
    "file": {
        "/etc/redis.conf": {
            "exists": true,
            "filetype": "file",
            "contains": ["bind 0.0.0.0"]
        }
    },
    "port": {
        "tcp:6379": {
            "listening": true,
            "ip": "0.0.0.0"
        }
    },
    "service": {
        "redis": {
            "running": true
        }
    },
    "command": {
        "redis-cli info": {
            "exit-status": "0",
            "stdout": ["redis_version"]
        }
    }
}

You can see the Goss config isn’t difficult to read but it’s quite syntactically dense. Due to the amount of Ansible work I’ve been doing lately I think I’m becoming more tolerant of this kind of config. I’m unsure if this is a good thing for me or not.

# and the comparable serverspec spec file

describe 'redis' do

  describe package('redis') do
    it { should be_installed }
  end

  describe file('/etc/redis.conf') do
    it { should be_file }
    its(:content) { should match /\s+bind 0.0.0.0/ }
  end

  describe service('redis') do
    it { should be_running }
  end

  describe port("6379") do
    it { should be_listening.on('0.0.0.0') } 
  end

  describe command('redis-cli info') do
    its(:stdout) { should match /redis_version:/ } 
  end

end

I personally find the serverspec tests easier to read and they benefit from being able to include comments. Swiftly moving past the personal choice of which you prefer, we come to one of the most novel Goss features, the ability to build a test config from interactive commands on a sample system. Below we’ll install goss on a test system and show how this works.

# install goss
# don't install software in this way on any host you care about

$ curl -L https://github.com/aelsabbahy/goss/releases/download/v0.0.4/goss-linux-amd64 > /tmp/goss
$ chmod +x /tmp/goss

$ goss --version
goss version v0.0.4

Now goss is installed we’ll build up a few basic tests. You can also watch a very short goss demo video that shows this and some other features.

$ goss add package redis
Adding Package to './goss.json':
{
    "redis": {
        "installed": true,
        "versions": [
            "2.8.22"
        ]
    }
}

$ goss add service redis
Adding Service to './goss.json':
{
    "redis": {
        "enabled": false,
        "running": true
    }
}

$ goss add port 6379
Adding Port to './goss.json':
{
    "tcp:6379": {
        "listening": true,
        "ip": "0.0.0.0"
    }
}

$ goss validate
......

Count: 6 failed: 0

As you can see from the snipped output after each ‘goss add’ command using it in this way builds up a working test suite based on current system state. You’ll want to edit the ./goss.json file and remove certain criteria, such as explicit version numbers, but for people new to infrastructure testing this can be an easy on ramp. It’ll be interesting to see popular this approach becomes, the RedHat emerging technology team had a similar tool for puppet many years ago called cft (if I remember correctly) which never really seemed to take off.

In terms of output the goss successful test output is currently quite bare, the ability to chose a more verbose output mode when the checks pass would be nice. Error messages from failed checks are more helpful but an issue in the goss.json config file can leave you scratching your head and reaching for your normal json validation tools.

# the addition of a trailing comma for example causes this
$ goss validate
Error: invalid character ']' looking for beginning of value

# and here's an example of checks failing
sudo systemctl stop redis

$ goss validate
..FF.....FFF..
redis-cli info: exit-status doesn't match, expect: 0 found: 1

redis-cli info: stdout: patterns not found: [redis_version]

redis: running doesn't match, expect: true found: false

tcp:6379: listening doesn't match, expect: true found: false

tcp:6379: ip doesn't match, expect: 0.0.0.0 found:


Count: 14 failed: 5

Update: newly added in Goss 0.0.6 is the addition of goss validate --format documentation. This new mode displays output verbosely even when all checks pass.

$ goss validate --format documentation
Service: redis: enabled: matches expectation: [false]
Service: redis: running: matches expectation: [true]

Port: tcp:6379: listening: matches expectation: [true]
Port: tcp:6379: ip: matches expectation: [0.0.0.0]

Package: redis: installed: matches expectation: [true]
Package: redis: version: all expectations found: [2.8.22]

It’s early days and I wouldn’t be able to replace all my serverspec usage with it just yet but for such a young project Goss is an impressive and worth watching possible alternative.