Most people use statistics the way a drunkard uses a lamp post, more for support than illumination. — Mark Twain

In an interesting article, “Know the Difference Between Your Data and Your Metrics,” Jeff Bladt and Bob Filbin illustrate the crucial “difference between numbers and numbers that matter” with the example of a YouTube video appeal for used sporting equipment which resulted in:

• 1.5 million views; but only
• 8 donation pledges and
• zero actual donations.

They argue that “... all metrics are proxies for what ultimately matters ..., but some are better than others.” The better ones they call meaningful metrics, to be contrasted with vanity metrics (such as the 1.5 million YouTube views) which look impressive but have no business value.

I suppose the moral of all this is the usual one: the usefulness of a tool depends not just on its inherent quality, but, more importantly, on the skill of the person using it. A sharp knife is more useful to a surgeon than a blunt one, but more dangerous in the hands of an amateur. Software metrics are a tool for IT managers, useful (or harmful) according to the wisdom (or naïvety) with which they’re collected and interpreted. Bladt and Filbin conclude:

Organizations can’t control their data, but they do control what they care about. ... Good data scientists know that analyzing the data is the easy part. The hard part is deciding what data matters.

So, where does this leave the (non-management) programmer? What is he or she to do with all these metrics? Probably — not too much. In some cases, of course, the programmer can increase his performance by aiming to improve specific, targeted metrics. For example, a programmer who is considered “slow” might aim to increase his output by monitoring weekly LOC, challenging himself to produce more code each week. But this quickly becomes counter-productive if the percentage of bugs increases. In that case, he has merely increased quantity at the expense of quality.

My conclusion is that metrics are useful for management, much less so for programmers. The programmer should, IMHO, focus on the task of producing quality software; in other words, on nailing down requirements, honing the design, and writing clean, self-documenting, well-tested code. Produce good code, and the metrics will take care of themselves.

Thanks to eyepopslikeamosquito for an interesting meditation.