My Girlfriend has had a lot of discussions at work lately, regarding metrics. So today when I used Graph your inbox, I noticed something funny.
I graphed the communications with my girlfriend in available in my inbox. I noticed that since august the sum per month of our communications is halved. How can this be interpreted?
Well it could be interpreted as something good, were talking more offline than online. It could be interpreted as something bad, were not communicating anymore.
Given just this metric as input we can make no confident assumptions about anything regarding our communications. Even if we can calculate this metric it has no value, because even if account the trend we can still say nothing about how well we are communicating.
Now there is this idea that metrics are a good way to drive a company, for different tasks it might even be the case. But in software development Metrics can often say what-ever you want them to! Or they focus on the wrong results.
So let’s look at examples when measuring can be good and when it can be bad.
At RemoteX we are measuring things as well. Among other things were using EQATEC to collect information about our applications. By collecting information about application crashes, we indirectly measure how often our applications crash. More importantly we also collect the information required to correct the crashes. As of the next release we’re measuring the time it takes for our offline capable applications to synchronize their data. And in the future we plan to measure how long it takes for users on average to finish certain predefined tasks.
What we are measuring is the quality of the product, we’re not measuring the process. The software is the end goal, not how many test cases were running. Not how many bugs we missed in the previous release, this is not important. What is important is to collect information in such a way that you can answer the following questions:
- How is the quality of our software right now?
- If something happens that lowers the quality of our product, will our collected information assist us to quickly fix the problem?
- Are the metrics collected measuring the product?
It is important to focus on the Product. The product is what you want to improve, the process is secondary. If you measure the process you will “improve” the process but the result of the process, the Product, will suffer because of it. The resulting product is the business value you want to measure, not the process.