Vanity metrics obscure true performance

‘Vanity metrics’ are any statistic that makes the business sound successful but do not inform stakeholders about actual performance or give insight into areas of growth (Ries, pg 143).

Excessive exclamation aside, these are examples of metrics which may be effective for the marketing team but give little insight into business operations. Entrepreneurs, whose obsession is to unlock growth for their venture, know this data isn’t useful to them. In their desire for success, and perhaps the sheer repetition as they share these metrics with dozens of investors; however, the entrepreneur can begin to rely on these metrics as business guides. They can forget to invest in the study of their customers and processes; the subjects their business is about.

The heroic stories of startups in America may appear haphazard and encourage excited entrepreneurs to ‘just do it’ (Nike). The reality of innovation; however, is that no one gets the idea right on the first go. When a business embeds valuable metrics into their processes from the start, they are equipped to make the inevitable pivots on the basis of sound data and customer interaction rather than gut instinct and fear. These metrics must have the three characteristics Ries describes: actionable, accessible, and auditable (Ries, pg. 143-146).

Actionable means the data can be correlated with business actions. Like a valid experiment, a good metric ought to control all variables except the independent. To put it another way, I ought to know whether it was my cold-calls that increased the number of first-time customers last month and not the new packaging the manufacturer put on the box. This is a goal to be aspired to rather than strictly maintained; however, since we don’t have strict control over the environment.

Accessible means the data is understood by all stakeholders. A metric that isn’t understood will be ignored or worse, may be used to validate behavior that has no correlation to business success.

Auditable means the data must be reproducible. Any metric which cannot be verified with another test run is of little worth and may even erode stakeholder’s confidence. Not only does it need to be capable of a second execution, it must be capable of verification against the people whose actions it’s measuring.

Every month kCura throws an all-hands meeting with a series of metrics about the business. It may be one of the most boring parts of the entire presentation for this reason: almost all the data qualifies as ‘vanity metrics’. The data no doubt excels at market leadership promotion but tells me nothing about how the actions of the business over the past month have impacted our bottom line. Perhaps the purpose of these meetings is to energize the company towards a single vision, and it performs this function well. Another avenue for valid metrics is not available to the common employee, however.

I recently wrote out my first Lean Canvas on a business idea our team plans to execute this Fall. Most of the data was easy to fill in, but I realized no conversations had happened (to my knowledge) about the metrics relevant to determine if our venture is successful. Therefore I will apply this insight as we continue to explore our current plan so we are able to determine the impact of our work from day one. I may also send feedback to the CEO about the prevalence of ‘vanity metrics’ ;)

References

TODO: connect this idea with David’s census