Smarter Service Status in Puppet

While most people know you can use puppet to ensure a service is running the mechanism it uses to determine if a service is actually running is often unexplored.

By default (at least up to Puppet 2.6) puppet assumes that a service doesn’t supply a working status option and so will look up the services name in the process table to check if it’s running. If your service does support the status argument you can set ‘hasstatus => true’ and the platforms service provider will be used to interrogate the services current status.

While most services only report a simple status of running or not running puppet, when you’ve specified ‘hasstatus => true’ puppet will consult a second property, if it’s present, - status - which is where things get a little more interesting and extendable.

  # puppet manifest
  service { "httpd":
      ensure    => "running",
      hasstatus => true,
      status    => "/usr/local/bin/puppet-status-http-check",
  }


  # puppet-status-http-check - example check

  #!/usr/bin/perl
  use strict;
  use warnings;

  my @checks = (
    "/usr/lib/nagios/plugins/check_procs -C httpd",
    "/usr/lib/nagios/plugins/check_http -I 127.0.0.1",
    "/usr/lib/nagios/plugins/check_http -I 127.0.0.1 -u /about",
    "/usr/lib/nagios/plugins/check_http -I 127.0.0.1 -u / -s udlab",
  );

  for my $check ( @checks ) {
    $check .= " 2>&1 > /dev/null"; # suppress output
    system( $check ) == 0 or exit 1;
  }

  # when running under debug you'll see a line like:
  debug: Service[httpd](provider=redhat): Executing '/usr/local/bin/puppet-status-http-check'

By specifying our own command in the status property we can do more complex, and domain specific, status checks. For example we don’t so much care that apache is running as that it’s serving our chosen vhosts correctly. You can use any command as the right hand side of status and puppet will treat a return code of 0 as confirmation that the service is running and anything else as a failure; which will trigger an attempt to restart the service in our example.

One possibility is to tie this in to nrpe-runner with a carefully chosen command name pattern to reap all the benefits of your already defined nagios checks.