- Work hard on finding the ‘perfect’ metric that measures the behavior you are trying to improve. Look for both the cause and the effect.
- In the absence of such a metric, do not ignore the ones that are only partially good.
- Do not measure something just because the data is available.
We have so much data floating around these days that the phrase ‘big data’ seems totally inadequate to me. We have a ton of data covering a variety of areas and it is growing at an ever faster pace. For instance, do you want to know how many people in the US did not take their allocated vacation time? There is data on that. Do you want to know what percent of books are bought on Amazon Kindle but are never finished (or, even worse, started)? There is data on that. Do you want to know how long it is taking other drivers to get out of the traffic jam you find yourself stuck in? Most definitely, there is data on that.
With so much data floating around, it has become easy to measure a lot of things. As the old saying goes, if you want to improve something, you have to measure it. This has led to a whole business of KPIs, Balanced Scorecards, and other metrics.
The key question is now that we have all this data and technology, are we measuring what should be measured? And, while we are on the subject, can what is really important be measured in every, or even most situations?
Let me use an example from my personal life. In December, I bought a popular wrist-worn fitness device. By default, it checks for 10,000 steps as the daily goal. Once I knew this was being measured, I noticed I did not mind taking a few extra steps to wherever I was going, proving to me to me that if you measure it, you can improve it. I remember once they had closed the access to a particular parking area at the Philadelphia airport. I got a look and a ‘you are funny’ remark from my colleagues when I shared my excitement on the prospects of meeting my step goals by walking through a couple of terminals. This counting of steps was all good until a few weeks later when:
- I figured out that height challenged people like me need to walk extra steps to get to the 5 mile goal that the 10,000 steps was intending to count. This made me change my goal to 11,250 steps.
- Then someone told me that counting steps is not very useful. It is the sustained (and ideally perspiration-inducing) activity that matters. To date, I have not found a way for my device to measure and report this (although that could be my ignorance as well). I have taken to using the count of active minutes as a proxy for this.
Let us now consider an example from the business world. Most would agree that the goal is to get happy and loyal customers. The question is: how do you measure that? One way is to measure the service level via fill-rate or on-time-in-full (OTIF) metric. But most companies are still measuring whether they shipped the material to the client on time, because this is the only data available to them. However, what should ideally be measured is whether the customer got the product on time and whether all the other aspects (product, quality, documentation etc.) were in order as well, kind of like a perfect order metric. However, this still does not capture the customer experience in placing the order and whether or not that was loyalty-inducing. A better approach might be to directly ask the customer (imagine that) about the service and how they track it and use that as the metric. So, it remains a heck of a job to measure how good a job one is doing in creating a loyal customer base.
While it is true that you have to measure something in order to improve it, it is not enough to measure what is easily measurable. In the absence of a perfect metric, it is still good to measure and improve a merely good metric. However, to make real change happen, one must strive for ways to measure what is really important or significant. This is of course easier said than done. In the business world, this has led to a lot of work around defining appropriate ways of getting to a good scorecard of metrics. Balanced Scorecard and SCOR are a couple of examples. A recent book by Lora Cecere titled Metrics that Matter also has a lot of good ideas in this regard.
Ultimately, it boils down to this. While all this data is good for generating metrics, let us make sure that we are generating the right metrics. Otherwise, it remains a bunch of well-meaning statistics, only partially useful at best and a complete waste of time at worst.