Metrics and their discontents

A fair amount of my job involves figuring out how to collect some data about work processes and... well, just packaging it in a table or chart so that managers can, presumably, use it to make decisions. Sometimes it's interesting because you have to get down into the esoteric nuances of the definitions involved in the data, debugging things along the way to get the data right and the right data. Most of the times it's just work.

But sometimes there's a little itch in the back of my mind that seems to indicate: maybe we don't need to do this. Or: maybe we're not measuring this right and someone who isn't us, whose work we're overseeing, is going to pay for it. Or: aren't we just expressing the same measurement in three different ways that say the same thing. And so on.

Measuring work is somewhat enjoyable, in the sense that you and your team can get better at what you do, but the time spent doing the measure sometimes also unironically eats the time you would spent doing what you do, which drives an unironic feedback loop of more measurement, less doing. It's frustrating.

There's an art involved in selecting what to measure, how to transform it, how to show it, who to show it to, how to use it, etc. I'm not an expert. But I think the most important question is: how am I going to use this? If the measurement doesn't have a specific use, throw it out. Go lean. Collecting for the sake of collecting is a disorder--hoarding. Side effects include: what's measured gets managed, even the data you collected that doesn't have a specific useful purpose. Don't believe it? Hand your boss a chart with a downward-pointing graph. Exchange your dignity for a few action items.

Fuzzy ideas, fuzzy ideas... I can't quite put my finger on what I think the problem is. In the meantime, a reference that for me has turned into a source of references, and a comfort when I think I'm the only person wondering why I'm carrying this bag of numbers around everywhere: Jerry Muller, The Tyranny of Metrics (2018) (notes).

Leave a Reply

Your email address will not be published. Required fields are marked *