Does Monitoring Alone Lead to Change?
Revisiting a blogpost from 2015. Sadly, I am no better at making 10,000 steps a day.
A few years ago, my husband got me a Fitbit. He wasn’t being rude; I asked for it. I am smart enough to know that just wearing the darn thing alone was not going to automatically lead to lost pounds. No more than just attending my Weight Watchers meetings each week leads to losing weight. It’s not surprising then, that I have not magically returned to my high school weight. So what’s the deal? Whatever happened to, “What gets measured gets done?”
You often hear people say that you have to measure the right things. I think it’s more accurate to say that you have to measure the right set of things. I can walk 10,000 steps every day, but I if eat lots of things that are high in calories I will be no better off.
Similarly, if we only track things that happen at point A and point C, ignoring the implementation of the intervention (i.e. point B), we may also miss the boat because we were hyper-focused on outcomes alone. In my example, if I don’t track the calories I put in my body (my Point B) along with the calories expended, I won’t reach my goal. I once heard someone at a shall-not-be-named very large federal agency say that a surveillance system is all that is needed. That attitude sounds like a cartoon that is familiar to most evaluators.
What happens in between points A and B are important. E. Jane Davison, in Actionable Evaluation Basics, advises that too often we get lost in the details and have no clear sense of what we really want to know. “Too often, evaluations don’t have any of these big picture questions; they leap to measurement or data gathering. Or they ask incredibly narrow questions, at the indicator level,” says Davidson. Monitoring alone is not necessarily an evaluative endeavor.
In my evaluation world, we deal with complex social problems. Measuring complex social problems is a challenge; measuring the effectiveness of our prevention efforts is even harder. We also have to decide what to measure at each stage of the intervention.
In the past, we have been guilty of collecting too much data that inevitably didn’t answer evaluative questions. We have clients who fall into the trap of measuring only what funders want to know. However, what funders want to know often doesn’t reflect “big picture” evaluation questions either.
Choosing what to measure is hard. One thing is for sure, monitoring alone does not lead to change. We have to measure the right things, at the right time, in the right combination. Walk on friends…