Tag Archives: metrics

What gets consumed easily gets measured

Leave a reply

What gets consumed easily gets measured, and what gets measured gets managed.

What gets managed, gets measured.

What gets consumed easily sometimes doesn't really mean much at all in the big picture but now it's the thing that gets measured, so you measure it.

And it gets managed.

Now there's a clear deviation between reality and the measurements, but what gets managed gets measured.

What gets measured gets gamed.

What gets gamed gets internalized in the unwritten tribal rules and regulations that exist somewhere just outside of what gets measured or managed, but what happens in Work Club stays in Work Club.

If this is your first time at Work Club, you have to perform as if you are working.

Attaboy.

Meanwhile, we observe with straight faces as that which gets easily consumed gets measured, and what gets measured gets embedded in a PowerPoint slide, and it's basically law at this point.

What gets embedded in a PowerPoint slide is inviolable, you are committing a thought crime right now if you think otherwise.

Thought crimes will be recorded in your permanent record and you will be questioned very sternly and publicly as a warning to all those who might be likewise inclined, and let's not even talk about your performance rating.

Meanwhile, in one teleconference or another, maybe all of them, it's hard to keep up, the measurements are pouring in and the managements are pouring out.

It's really beautiful if you look at it from the right angle—like a great migration across some distant savanna, like salmon splashing up a river to spawn.

I didn't get a "harrumph" out of that guy.

We have to protect our phony baloney jobs.

What gets managed easily, gets measured.

What gets measured, gets consumed.

What gets consumed, gets excreted.

What gets excreted, gets consumed.

What gets measured, gets the whip.

What gets measured is the whip.

I am Jack's smirking metrics.

Harrumph.

Sports numbers

Leave a reply

Trailhead: Tim Britton. Prospectus Q&A: Rich Hill, Ace Pitcher. Baseball Prospectus (2015-05-23).

“I don't dive all in to every stat. I think you can do paralysis by analysis. But if you can say one thing will help this guy realize that, 'Hey, your curveball is your best pitch. If you throw that at a higher percentage, you're going to be more successful,' why wouldn't that be something the player wants to hear? It benefits the player, benefits the team, benefits the organization. You take it for face value and understand what it is. I really believe if you can have the player understand what some of the numbers mean—not all the numbers, just some of the numbers—you can improve the player. Even if it's by one percent, you're getting a lot better.”

Metrics. Even in sports—especially in sports.

But this description of metrics catches two things that we don't do well on the job sometime:

  1. We often don't tell the person being measured what the numbers mean and why we're doing it. It's for the people above, not the people below.
  2. When we provide metrics to the people doing the work it's often "well here they all are". It's an incoherent pile.
  3. The metrics are "how many widgets did you make", not "how well did you make the widgets", so it doesn't necessarily lead to better performance, but maybe more things performed

None of those complaints are universal or constant, but they exist. I often lean on sports and it's metaphors or things like leadership and training. But I'm wondering if they're way ahead of us in our technical industry in terms of technical metrics—or at least meaningful ones. It could be that I'm just missing out (they seem to have good dashboards on the manufacturing floor), but I'm also thinking that it's worth stealing metrics from baseball and other sports—either literally or as analogs—and bringing them to work. At the absolute least it could inject some fun into what we do.

Alice's adventures in metricsland

Leave a reply

I was looking for something else entirely when I ran into this sequence from Alice's Adventures in Wonderland:

“Would you tell me, please, which way I ought to go from here?”

“That depends a good deal on where you want to get to,” said the Cat.

“I don’t much care where—” said Alice.

“Then it doesn’t matter which way you go,” said the Cat.

“—so long as I get somewhere,” Alice added as an explanation.

“Oh, you’re sure to do that,” said the Cat, “if you only walk long enough.”

Since I was thinking about metrics at work around that time, the correlation seemed obvious to me. I think we've all been there if we have to get our work performance measured: the boss wants you measure something, measure anything, just bring a number or, better yet, a chart with the line pointing up and to the right.

Is that the thing you should be measuring? Is that value you're tracking meaningful? Helpful? Is it going to be gamed by your workers because they know that it's (a) neither meaningful nor helpful and (b) they get rewarded if it points up and to the right and punished if it doesn't? Oh, you bet it will. And, in a sense, it should. It's a Catch-22. Only insane people would behave as if an insane system is sane, whereas deviation is a sign of sanity.

A quip from Marilyn Strathern (the de facto version of Goodhart's Law): "When a measure becomes a target, it ceases to be a good measure."

Here's the part I was originally looking for because I think it's also project related:

“But I don’t want to go among mad people,” Alice remarked.

“Oh, you can’t help that,” said the Cat: “we’re all mad here. I’m mad. You’re mad.”

The worship of the yardstick

Leave a reply

I just read this article (the text of a presentation, really): James L. McConaughy, The worship of the yardstick, Educational Review, Vol. 55, No. 3, March 1918, pp. 191-200

(Yes, that is: nineteen eighteen.)

How did I get there? First, I read The Tyranny of Metrics, which cited Education and the Cult of Efficiency, which cited this article. (I like to follow the citations because I'm a Super-Fun Guy.) All three of them, for me, have ineluctable titles—tyranny, cult, worship—that leave little doubt about which direction they're approaching from. Moreover, they're all related to a problem that drives me crazy nearly every day: the unquenchable, unquestioning love for measurement at work.

It's out of balance. On a recent project, metrics (the word that means "measurement" at work) accounted for nearly 50% of our group's time—so much so that it was necessary at the culmination to bring in extra people to actually do the work. Here's a natural question to ask: did the work get done? Of course not. Mission Accomplished.

Here's where I drive people crazy: I push back. But I think I'm misunderstood. Because I'm such a smooth talker when I get worked up. I'm not against metrics or measurement—I use measurements unprompted in my own life—but rather the mindless way in which they're applied. I get upset when the project bureaucrat asks why a document is only 84% done when the plan clearly states that it should be 87% done. It's like someone knocking on the bathroom door, upset that you're only 84% done taking a dump when you've clearly been in there long enough to be 87% done. What? Anyway. It's not a real measurement. There is no such thing as an 84% done document. It's a heuristic. It's not real. But measurements become very real when everyone pretends they're real. And measurements become the standard against which rewards and punishments are meted out. After a while people forget that the numbers aren't real, and they don't think very deeply about what the measurements mean. And they don't have any feeling in their hearts for the product itself, which is just some sort of abstract byproduct of the process.

I will die on this hill.

Here's one other worry that I have about the cult of metrics. Whenever I get saddled with some new measurement, I try to find a way before the end of the day to automate the calculation. I get to do a little bit of thinking while setting up the algorithm, and it costs me less time over the long haul. Win-win. But what becomes of the people who every day, every week, etc., devote non-negligible amounts of their time to manually calculating things that are easy to replace with a program? I guess we'll find out.

Data fictions

Leave a reply

Those who gather and interpret such aggregate data understand that there is a certain fictional and arbitrary quality to their categories and that they hide a wealth of problematic variation. Once set, however, these thing categories operate unavoidably as if all similarly classified cases were in fact homogeneous and uniform.

—James C. Scott, Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed, Chapter 2: Cities, People, and Language (notes)

This is one of the aspects of work that I really try to press on younger engineers: metrics are just metrics. Especially in organizations that live and die by earned value management, it's easy to forget: the map is not the territory. Metrics are just metrics. People get hung up on the difference between, e.g., a specification being 82% done and 83% done. It doesn't matter. It's the trend that matters. The specific number is just a heuristic. It has no meaning as a precise measurement like the outside temperature or your tire pressure. It's just a formula that I applied to a database based on a tailoring of a company-wide process instruction with weights that I applied based on experiences (read: bias).

But the real problem is that once you put that heuristic into place—that definition or interpretation of the data that everyone knows at the onset is a useful fiction—it tends to become Law. The old people forget or move on, and the new people never knew. Whether out of fear or custom, one thing is certain: the Law must be upheld. What happens to the person who tries to remind everyone that, hey, wait, that's not a law, that's just a helpful fiction? Enjoy wearing that big scarlet A on your chest, Asshole, it matches your eyes.

Metrics are like your speedometer reading: glance at it when you need to know, but don't stare at it or you'll really be in trouble.

Anyway, Matthew Mcconaughey does it better:

Fugazi, Fugazzi. It's a wazzy, it's a woozy. It's [whistles] fairy dust. It doesn't exist. It's Neverlanded. It is no matter. It's not on the elemental chart. It's not fucking real.