Rocket Surgery

The movie The Untouchables had Robert DeNiro as Al Capone deliver a great line: “You can get a lot further with a kind word and a gun than you can with a kind word.”

Now, I’m not promoting the violence, but in the world of corporate computing there will always be times of competing claims. You’ve seen it. The systems guys will claim your application crashed their database. Another developer will claim that you’ve broken their code. Your project manager will claim that you should be able to complete a task in a certain amount of time that they’ve pulled out of thin air. You obviously think differently, but how do you make your case?

Well you can get a lot further with a kind word and a gun than you can with a kind word, and that gun is a number. A quantitative measure. In short, a metric.

Now, many developers don’t like metrics. After all, metrics aren’t coding. We developers like churning out the code and drawing UML and refactoring to patterns and quoting Josh Bloch. Metrics are for the project managers, right? Well, I say the metrics are for you, the developer, because they promote transparency and accountability.

Transparency makes your job easier because the more transparent the organization, the easier it is to obtain the information you need and the harder it is for others to conceal it. This includes both technical information (like the health of the code, the progress of the project, the number of defects) and meta-technical information (like requirements). There’s no hiding in a transparent organization.

Accountability makes your job easier because the more accountable people are, the more honest people have to be. In highly accountable organizations, you have the luxury of trust. As developers this means you are more likely to rely on each other and less time doing things over yourself because you know you’ll each be held accountable for your own work.

So, what if you’re in an organization that doesn’t have a metrics program? Start your own! Start collecting some metrics on your own. What if management doesn’t really show support? Collect them anyway for yourself. Eventually you’ll be able to whip out your number along with a kind word and the value of them will show themselves.

A friend who is a Lieutenant Commander in the Navy tells me they have a saying, “You get what you inspect, not what you expect.” Even if you’re the only one being measured and seeing the numbers, you’ll know where you stand, and you’ll be setting goals for yourself and know if you’re improving or not.

Here are some metrics that I highly recommend keeping:

Defects (counts, severity, longevity)

If you collect no other set of metrics, collect this set. If you don’t have a list of known defects - and I mean one that’s kept in a storage system other than someone’s memory - GET ONE. Bugzilla is free. I’ve used phpBugTracker before with great success, and it’s also free. You can also be effective with a spreadsheet on a shared drive. But if you’re not tracking your defects, how many you have, when they were detected, how severe they are, who’s working on them, and when they were resolved, you aren’t managing your software.

Code Coverage

How much of your code actually gets executed when you run your unit tests? (You are running unit tests like JUnit or TestNG, aren’t you?) Higher numbers are obviously better, although high coverage does not necessarily mean the application is bug-free (another topic). Emma is my favorite tool to collect this metric, but jcoverage/gpl is another free alternative (quite a bit slower, though).

Defect Removal Efficiency

How many bugs did we developers find and remove compared to the number of bugs that were found in production? Higher is better. Just count each defect that you find and correct — this is R. Then count the number of defects reported in production — this is P. Your Defect Removal Efficiency (DRE)is R / (R + P), and is expressed as a percentage, like 80%. You can then make statements like “We removed 80% of the known bugs prior to going to production.”

Hours Spent on Task

You’re probably reporting this number on a timesheet anyway, although it might not be fine grained. Hang on to the numbers.

Scope of Task

This one will be project and environment specific, but look for a way to measure how big a task is if there isn’t one already. For example, if a task requires you to create 3 web pages, modify 2 web pages, and add a database table, then there’s a pretty good idea of the scope of the task.

Those last two obviously go in pairs. Eventually, somebody will ask you to estimate how long it will take to do a task. Would you rather guess, or guess after looking back at how long it took you to complete a task of similar scope? Or when your project manager low-balls an estimate and wants to know why it’s taking you so long to complete something, wouldn’t it be satisfying to show quantitatively that the last three times a task of the same type was completed it took five times as long as their estimate for this task?

So, in summary, I would urge programmers to heartily participate in their metrics program if there is one at their workplace. If there isn’t one at their workplace, try and get one started. If you can’t get traction, keep your own metrics. You’ll get farther with them. (Don’t forget the kind word.)

This entry was posted on Tuesday, February 7th, 2006 at 2:59 pm and is filed under Software development.You can follow any responses to this entry through the RSS 2.0 feed.You can leave a response, or trackback from your own site.