Tracking What’s Not There

A common theme among the Centers for Medicare & Medicaid Services (CMS) value-based programs is reduction or lack. The Hospital Readmission Reduction Program (HRRP) measures value by readmissions that don’t occur. The Hospital Acquired Conditions (HAC) reduction program measures value by hospital-acquired conditions that never set in. And so on.

Tracking events is easy. Tracking non-events is harder.

Even talking about how to track non-events is hard. A Google search for “how to track what’s not there” gave me dozens of results on tracking missing Apple devices, packages, and how to not to be tracked online. “How to quantify what’s not there” returned pages of results interrogating the business adage “What gets measured gets managed.” “Tracking non-events” taught me a lot about Google Analytics’ “non-interaction events” but not much else.

The challenge of tracking what doesn’t happen plagues every field. One of the hardest parts of addressing “stop and frisk” reform, for instance, is that every recorded court case involves a stop and frisk turning up guns or contraband. Of course it does. If the cop hadn’t found anything, there wouldn’t have been a case to take to court.

The fact that contraband turns up in case after case doesn’t mean stop and frisk rules are “good” or “work” or are Constitutional. Yet it’s too easy for us to jump to those conclusions because we cannot see what’s not there.

Or consider the classic case of survivorship bias:

Reinforce your aircraft based on where they got shot in a firefight, and you won’t actually help your pilots – because you’re focusing on where they could get shot and still make it home, not where a shot is fatal.

Likewise, data on hospital readmissions or hospital-acquired infections doesn’t usually account for all the people who didn’t return to the hospital or didn’t pick up an infection while hospitalized. The numbers look at what’s there, but they may miss the context.

But acquiring context, frankly, sucks.

Context requires more data. As if we didn’t already have enough data. As if every single office on the planet doesn’t already have more data than it can use – and also, somehow, not enough data on the things it really wants to know. Like how many people aren’t dying of MRSA this week.

I’ve been interested in what’s not there ever since I realized that on any given Thursday, at least as many white people in my middle-class suburb are probably doing illegal drugs as black people in the “wrong side of the tracks” neighborhood just north of downtown. In fact the latter number may be lower, given aggressive drug testing requirements unequally imposed on the lower classes and the fact that drugs generally require one to have money left over after paying for survival.

Yet we’ll never know. Instances of illicit drug use in this city are recorded by police interactions. I haven’t seen a police car in this neighborhood since the time my neighbor’s garage caught fire in 2006.

How do we track what we don’t track?

---

Let me know if you read this by tossing a couple dollars in the tip jar.