Data fictions

Those who gather and interpret such aggregate data understand that there is a certain fictional and arbitrary quality to their categories and that they hide a wealth of problematic variation. Once set, however, these thing categories operate unavoidably as if all similarly classified cases were in fact homogeneous and uniform.

—James C. Scott, Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed, Chapter 2: Cities, People, and Language (notes)

This is one of the aspects of work that I really try to press on younger engineers: metrics are just metrics. Especially in organizations that live and die by earned value management, it’s easy to forget: the map is not the territory. Metrics are just metrics. People get hung up on the difference between, e.g., a specification being 82% done and 83% done. It doesn’t matter. It’s the trend that matters. The specific number is just a heuristic. It has no meaning as a precise measurement like the outside temperature or your tire pressure. It’s just a formula that I applied to a database based on a tailoring of a company-wide process instruction with weights that I applied based on experiences (read: bias).

But the real problem is that once you put that heuristic into place—that definition or interpretation of the data that everyone knows at the onset is a useful fiction—it tends to become Law. The old people forget or move on, and the new people never knew. Whether out of fear or custom, one thing is certain: the Law must be upheld. What happens to the person who tries to remind everyone that, hey, wait, that’s not a law, that’s just a helpful fiction? Enjoy wearing that big scarlet A on your chest, Asshole, it matches your eyes.

Metrics are like your speedometer reading: glance at it when you need to know, but don’t stare at it or you’ll really be in trouble.

Anyway, Matthew Mcconaughey does it better:

Fugazi, Fugazzi. It’s a wazzy, it’s a woozy. It’s [whistles] fairy dust. It doesn’t exist. It’s Neverlanded. It is no matter. It’s not on the elemental chart. It’s not fucking real.

Leave a Reply

Your email address will not be published. Required fields are marked *