Introduction To DSAC

A while ago @ripienaar and I had a chat in a pub about monitoring, event systems and lots of related subjects. As we all know he’s way more productive than is fair and so while he’s been doing a BUNDLE of work with on subjects like monitoring frameworks and event correlation I’ve been doing some thinking (and no actual coding) about event auditing, continuous compliance and security event management.

Now I’ve finished the $TIMESINK_PROJECT I’m soon going to actually need some of this stuff so I’ve started putting together a prototype framework that I’m calling DSAC - Dump Send and Correlate. The code is in a very early stage at the moment but is dealing with a small number of agents on a test network of a couple of hundred nodes. I’m going to start documenting the sections as it becomes ready for more public consumption but I thought I’d show my architectural plans for version 0.1.

The architecture is quite simple at the moment. Every node runs the “consumer and dispatch” stack which generates events, currently all events are made from cron invoked agents. A separate process, also cron invoked (for now) then runs through the spool and invokes all the dispatchers that have registered an interest in the output of that agent. Simple dispatcher examples are an AMQ pusher or a MySQL loader.

At the other end of the process, and quite symmetrically, we have the consumer stack. This reads from the nice big fuzzy cloud of transient data loss and spools files for later processing. We then have another process pick the files up and run them through a number of processors.

I’ve got working prototypes of a simple bulk archiver and some debugging aids but I can also envision some more useful real time dashboards. The last stage at the moment are the simple reports. I’m currently focusing on the easier reports that will help me show changes to an auditor, package updates, service status changes and user logins but this step will hopefully expand to encompass a lot of our rote compliance needs.

Once I’ve tidied up the code (and picked up some more ruby!) I’ll start putting the bits I work on in my spare time on github.