Why do we need metrics? My suggestion: to make better decisions.
Any metric we use should be either an experiment (we think x will happen if we get metric y, but we don’t know, so let’s test it and learn) or it needs to be chosen specifically to support a known decision.
If a metric doesn’t serve a decision, why would we waste people’s time asking them to understand and build a relationship with it?
A few months back we made an unintentional discovery. In order to clean up some dashboards we asked: what is the most important decision you need to make and then what are the metrics to help you make that decision?
The discovery: the teams didn’t come up with metrics. They came up with models. When you know what improvements you want to make — which are the outcomes of your decisions. Then why would you steal precious time away from a human to make that improvement if you can just train a computer to do it for you?
The human interprets the world, designs the question and the model, and the machines perform the improvement.
So in some cases I would argue: metrics are for machines, humans need models.
Then question is: when?
