Dean Wilson@UnixDaemon: ... It's not even close to relevant.

Disturbing Diffs - Unsafe open?


-  file_move_safe(move_from_path, move_to_path)
+  move_file(move_from_path, move_to_path)

Is move_file not as safe as file_move_safe? Is it safer? Dare I read the other diffs to find out? Am I better off not knowing?

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!

Events - November 2008

It's actually a good month for dynamic language fans in London as we've got both the London Perl Workshop and the inaugural Ruby Manor - both of which I'll be attending.

Although, as a sysadmin, I feel a little bad about not making it to the Linux 2008 event (organised by the UKUUG) I couldn't really justify the time and cost this year. The talks were a decent selection but not enough to get me up to Manchester on my own budget for a weekend. I'll have to keep an eye out for next years LISA event in London to make up for it.

Last but not least - the FOSDEM 2009 dates have been announced (for the second time). Assuming they don't change during the week I'll be booking those before Xmas. Roll on February!

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!

Dynamic Languages and joining arrays

I've been spending a fair amount of time recently trying to choose my Language of the year for 2009. I've always been a dynamic language fan (yes, I know this means I should be looking further afield for the next one) and I was surprised at how different even such a common task as joining all the elements of an array together, using a given separator, looks between them.

First let's look at the big three, including perl, my current favourite.

  
# perl
$ perl -d -e 1;
DB<1> my @names = qw( A B C);
DB<2> print join(" : ", @names), "\n";
A : B : C

# python
>>> names = ['A', 'B', 'C' ]
>>> " : ".join(names)
'A : B : C'

# ruby
irb(main):009:0> names = [ 'A', 'B', 'C' ]
=> ["A", "B", "C"]
irb(main):010:0> names.join(' : ')
=> "A : B : C"

The perl approach is very procedural (ignore the use of the debugger as perl doesn't come with an excellent REPL in the core like the other two) and is the one I'm most familiar with so it's hard for me to be too critical about it. If you like OO then it's not for you.

Next we have Python, which is really growing on me as a language - apart from in this case. Putting the separator first and passing the list in as a parameter just feels very wrong and is the exact opposite of the ruby version, which I much prefer. To me the ruby approach of operating on the array is the most natural version and sits well in my head. As a small 'bonus' I also looked at the PHP equivalent -

  
# PHP
$array = array('A', 'B', 'C');
echo implode(" : ", $array);

This is close enough to the perl version that I can't really object to it, other than to (rhetorically) ask why the hell it's called 'implode'?

I guess all I can say in summary is round one to ruby.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!

Rebooting Via Proc and the magic sysreq key

You know what the best way to start the day is? I'm pretty sure that it doesn't include a production web server putting its file systems in to read only mode. When this happens most local commands don't work - init, shutdown, telnit and reboot all stop being useful and you have to resort to desperate measures... and here's the desperate measure of the day.

First, check that your system supports the magic sysreq key -


$ cat /proc/sys/kernel/sysrq
1  # nonzero is good

Now you know you have the power to destroy your system through a single incorrect character, have a look at the Redhat Sysrq command reference (you want the 'sysrq' section). We tried to make it sync the disks and reboot - your requirements may vary.


root@web02:~# echo s > /proc/sysrq-trigger
root@web02:~# echo b > /proc/sysrq-trigger

# machine reboots

As techniques go this one's a little obscure but it's very useful in the right circumstances.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!

October London Python UG

I made it along to my first ever London Python User Group tonight, and from what the regulars said about the turn out so did a lot of other people. Over 50 people in attendance is very respectable.

The first talk was a bit of a let down, it felt really long, quite slow moving and could have been much better as a lightning talk. Shame it was the best part of over an hour. Luckily the lightning talks themselves were good. Even though I'd seen a couple of them before at PyCon UK. PySmell, which is actually an IDE intellisense / auto-completion helper rather than anything to do with refactoring, is interesting (and you can read the slides online) and Metaclasses in Five Minutes (which took seven minutes) were both highlights of the evening.

ThoughtWorks have very nice offices in London (with a great view) and I'm looking forward to the next one. Kudos to Simon Brunning for organising it and let's hope Leon has the same turn out for tomorrows London.pm tech meet.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!

The answer might be 'it depends'

You're in charge of a server that provides two types of assets. The first type is public and its visibility is important to your company. The second should be restricted access only and shouldn't be public.

Now suppose there is a mistake made and the private material is exposed publicly - what's more important, that the public data is available or that the private data isn't? Who'd make that decision where you work? How long would it take to get an answer from them?

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!

PyCon UK - 2008

At $DAYJOB I'm working with a strong team of Python (and Django) developers so over the last couple of months my interest in the language has grown. Thanks to YAPC::EU not being very exciting this year I had a spare slot in my "conference schedule" and went to the highly recommended (by me and previous attendees I'd spoken to) PyCon UK. I'm glad I did.

I was more than a little out of my depth in most of the talks but a lot of the speakers were excellent, especially Raymond Hettinger - who I ended up stalking (by accident) and seeing all of his talks. The technical level required of the audience was quite varied but I ended up going to a lot of the more technically indepth sessions as they just seemed more interesting. The downside is that I lacked the ability to filter module based talks in the same way I can at Perl Conferences and that I learned (the hard way) that Python has many test frameworks, modules and harnesses.

The venue itself was fine, large, easy to get around and had restaurants and pubs near enough that you could make a dash outside for lunch. The keynotes were both very interesting - Mark Shuttleworth and Ted Leung both gave their view (in different ways) on where Python is, was and should be. As a (mostly) Perl guy I was a little surprised by how little it even got mentioned - twice by my count and each time it was as an afterthought. In a way this is reassuring, it fits in with my own views and encourages me to learn a new dynamic language (Python and Ruby are both interesting in different ways).

I should probably note that There won't be a PyCon UK in 2009 - instead the organisers are doing PyCon Europe 2009. And based on how good a time I had this year I'll be there.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!

Google Dev Day - London 2008

I recently went to the London 2008 Google Dev Day (the title of my post doesn't lie!) and while it was lovely to be near that hallowed grass (only half of which was actually down) the talks themselves left a lot to be desired - actual technical content.

I'm not sure if I'm the wrong audience in that I've already looked at the front pages and the code samples but I hoped, given the word developer in the events title, that it'd be a bit more tech heavy.

The actual talks were mostly well presented but they lacked any real depth on the subjects, most of them contained very similar material to the actual API introductions. It was nice catching up with some ex- coworkers though and, if nothing else, I've been inspired to look at the Google Visualisation APIs a lot more. When I get some spare time. Still, here's something to tick off on my PiP.

You can also view the Google Dev Day Videos on YouTube.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!

Spooks Code 9 - Making Torchwood look Good

When it comes to spinoffs the BBC isn't doing too well. After two, very, very bad series of Torchwood we're now 'blessed' with Spooks: Code 9. It's got nothing to do with the main Spooks series (a series I do like), has very... inexperienced acting and dull plots.

What's good about it? A lot of the cast are very pretty and it's only 6 episodes long. Luckily it's been panned by nearly everyone who's posted a review of it (I like to be on the bandwagon every now and again) and with a little luck it'll be canned after just one season. Consistently bad. Let's hope this one stays dead.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!

My First Day with Python - Initial Thoughts

While I've always been a bit of a perl guy I don't want this post to be "perl has x and python doesn't" in tone. Which is lucky really as Python has exceptions and threading as first class features where as perl has... ahem.

So after spending a chunk of today reading a python book and spending some time writing code here's my initial short list of gripes -

except IOError
print adding newlines
Significance of whitespace in blocks. But not like that.
The lack of ++

Considering how picky I can be that's a very short list so Python must sit well with me so far. Now, in order, I can't help but read except IOError as 'catch everything apart from IOError'. This one bugs me more than it should but considering how happy native exceptions in the language made me this just felt mean.

Secondly, print adding newlines. While this might seem trivial every other language I use on a daily basis has a print function that doesn't print a newline so this feels weird. At least it's not called say ;)

Now to the one that I'll get no sympathy on - whitespace in blocks. First up let me say I don't mind about the enforced indentation. I indent anyway so it's not a big deal. I guess I'll hit the odd case when it annoys me (probably involving heredocs) but I've got nothing against it. What does irk me is the lack of block delimiters - whitespace just doesn't cut it for me.

I like my { and } delimited blocks, a nasty voice in my head is telling me to add them but just comment them out ( if x == y: # { ) but that seems very wrong. I've always looked at those examples in C programming books that say...


# incorrect
if ( something )
  print("All's well");
  wellness++;

# this is wrong because wellness is a separate statement
# and not part of the if

... and thought - "just add the damn braces, you'll be back to add more code later anyway." Now I'm learning a language that seems to want me to slip up like this. I'll either get used to this or move to ruby.

Lastly we have the lack of ++ and --. I know the arguments, I've read them before. I disagree. I've never done anything insane with ++ and where I have used it it's saved me typing. Can we have ++ and remove nested ternary ( ? : ) instead please?

I like Python and I think I'll be investing more time in to learning it.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!

Pragmatic Investment Plan - End of 2008

In the past I've written up a small list of general goals to help measure my technical progress. Over the last few years I've become a lot busier and this habit fell by the wayside. But no more! I've got a quarter left and I'm going to try and complete...

Write and publish a technical article.
Attend two technical events.
Read and review 3 books.
Write and publish two Perl modules.
Create a personal Debian repo
Create 4 Debian packages, at least one of which should contain other peoples code.
Write 30 blog posts - at least 15 of which should be technical.
Choose the programming language I'll be learning next year.

Considering this is one of the busiest times of the year I have no idea how far I'll get but I do think it's worth at least an attempt.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!

Ubiquity - More Than Just Shiny Chrome

While Google Chrome has been getting all the press coverage recently Ubiquity, from Mozilla Labs, is where all the interesting action seems to be happening.

Ubiquity ticks all the boxes for me, it's a simple, easy to use idea, that'll save me time. It's easily extensible and already has a huge community of people working, enhancing and just trying new things with it. All the things I've come to expect from Firefox and the Mozilla using community.

I personally think this is an important distinction to make - while Google Chrome is a new browser with some great ideas (and a quickly revised EULA) FireFox is a proven, Free platform that encourages extension and has a track record of doing the right thing.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!

Google Chrome - Initial Thoughts

Like most of the techy part of the Internet I dutifully downloaded Google Chrome today and had a little play around. And just like all those other people I'm going to write about it. The difference is I'm very ambivalent about the whole thing.

Chrome seems nice enough. It's quick, works with all the websites I've tried so far and does have a killer feature - the task manager. Finally breaking tabs out in to their own sandbox is an idea whos time should have come years ago. Being able to see which sites are doing hugely evil things with my memory is a wonderful thing. I'm also inappropriately happy with the in-page search showing how many matches it found.

Unfortunately that's about it. While the minimal design and streamlined core functionality are lovely, these days I'm used to my extensions - the web developer toolbar, YSlow and the work flow changing Ubiquity are just too useful for me to give up.

It's not just the fact that these extensions are missing that puts me off, it's the lack of how to write custom extensions, searches etc. that feels wrong. Firefox is a platform as much as a web browser. Using Chrome what is the command line for pulling out the memory usage for the currently opened tabs? Do I need to screen scrape a running about:memory? I can't help but think they'd have three Firefox versions ready for download by now.

So will I be moving over to the new and shiny? Not yet. As useful as the broken out tabs are I need more functionality than Chrome can give me, so while I might use it for some day to day surfing it's no where near ready for me as a developer. Although I;m guessing they never intended for it to be.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!

Nagios Service and Hosts stats - Graphed in Munin

We've been hitting some load issues on one of our monitoring machines recently and while it looks like the munin graph generation is the culprit we also decided to keep an eye on how many services and hosts Nagios was checking.

One of the downsides of having a very automated server deployment system is how easy it is to suddenly find yourself with an extra dozen hosts you no longer really need. While each check is quite small and quick, add up the frequent runs and multiply it by a reasonable number of servers and you can soon hit problems.

So as a first step towards keeping an eye on those numbers we now have a munin Nagios hosts plugin and a munin Nagios services plugin that show the total number of hosts and services monitored and the states those resources are in.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!

Nagios Checks - Validate HTML and Validate Feed

As part of my ongoing attempt to stop myself from silently making mistakes (I don't so much mind the ones I notice) I've added another couple of Nagios Plugins. This time validate_feed and validate_html.

As both of these checks call out to an external, third party resource, if you use them be sure to tweak your Nagios polling interval down to a respectful level.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!

UnixDaemon.net gitweb - because everyone else has one!

I'm not exactly a demanding user of version control systems so I've not been heavily motivated to ditch my personal SVN repo (which I don't use as much as I should) and plunge in to the shiny new distributed ones. However (and this is my excuse) I've recently wanted to put a handful of my own Nagios plugins under a public VCS. While we use a number of the checks at work I don't necessarily want the local changes to be made immediately public so I thought I'd take this as an opportunity to have a fiddle with git.

I've now got my own gitweb instance (because I'm a tech sheep) and while it's pretty easy to install and setup it was oddly difficult to track down how to modify anything beyond the basics (the answer? Hack the feature hash from the config file - eg $feature{'snapshot'}{'default'} = [];). First impressions of git? I've got a lot to learn. The basic commands were easy enough to pick up but I am very conscious of how little of its power I'm using.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!

The Mummy: Tomb of the Dragon Emperor - Short Review

It's been the summer of returns, from the very enjoyable Dark Knight to the should have been left buried 'plot' of Indy and the Crystal Skull. Unfortunately most of them have been rubbish - and the The Mummy: Tomb of the Dragon Emperor doesn't do anything to address this.

The special effects are good but nothing ground breaking, the new Evelyn O'Connell is a masterpiece of terrible - how such an uninteresting character can steal and kill so many scenes baffles me. The story is predictable, boring and lacks the hammy goodness of the first two. At least they didn't cast Shia LaBeouf. 3/10.

Now roll on Hellboy 2!

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!

Filter syslog logs with syslogslicer

While digging through a pile of syslog log files recently I needed something a little more data format aware than pure grep. So I present the first version of syslogslicer - a simple perl script that knows a little bit about the syslog log file format.


 # some example command lines
 syslogslicer -p cron -f program,message /var/log/syslog
 # print the program and message for all lines with program 'cron'

 syslogslicer -p cron -m hourly /var/log/syslog
 # all fields for all lines with program 'cron' and message 'hourly'

 syslogslicer -p cron -m hourly -s 20080810100000 -e 20080810123000 /var/log/syslog
 # all fields for all lines with program 'cron' and message 'hourly'
 # between 20080810100000 and 20080810123000

syslogslicer allows you to filter the output by matching text in the program or log message, only print certain output fields and do basic time based filtering. If you've ever wanted to see all the logs raised by postfix with the word 'database' in them between 10 and 11 am then this might be the tool for you.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!

Nagios - Check Proxy Check

"This script retrieves a URL via a specified proxy server and alerts (using the standard Nagios conventions) if the request fails."

We're running a couple of services through a proxy server for a number of good, and to be honest a couple of not so good but mandated, reasons. The Check Proxy Check Nagios Plugin ensures that if the proxy goes down in a way that stops us pulling pages through it we know.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!

Apache JMeter - Short Review

A short review for a short book. Apache JMeter (Packt Publishing) is a good book if you're new to both IT and testing and want your hand securely held. It introduces you to the basic ideas behind automated testing, takes you step by step through some simple GUI test cases and then doesn't go any further.

It's a short book and maintains its beginners focus well but it has a very short lifespan (luckily it's also available as a cheap PDF) and if you're comfortable with GUIs and basic testing, or willing to click around for a while I'd recommend you dive straight in to the JMeter GUI rather than investing half a day to read this book.

On the downside it didn't cover any of the aspects of JMeter I found interesting and wanted to learn about - the access log sampler and distributed load testing spring to mind - which in a beginners book is fine enough but does make it completely the wrong book for me.

Like this post? - Digg Me! | Add to del.icio.us! | reddit this!