Every once in a while, every manager in every organization gets the brilliant, yet uncanny, idea to measure the productivity of software delivery—or, to put it more precisely, to measure the individual productivity of software developers.

Not long ago, standing outside in the early spring sun, I enjoyed a pleasant conversation with the COO of one of my clients. Again, measuring productivity was the topic, as in many similar conversations I’ve had with managers over the past thirty to forty years.

Counting commits

“I think we should be able to measure who’s good and who’s sufficient, or even not so good,” the COO philosophized. “What if we count the number of lines of code they write,” and seeing my surprised reaction, “or perhaps better, the number of commits they do. That should give us a clear indication of their productivity, right?”

Well…

“There are a few things you should consider,” I replied. “First of all, the code being written only covers a small part of the time spent by software developers. Most of the time goes into discussing matters with the business, contemplating and designing a solution, which mainly takes place in their heads, and let’s not forget meetings and stand-ups.”

“Secondly, and I think even more importantly, developing software is not an individual sport, like tennis or even chess; rather, it is typically a team sport, more akin to volleyball or hockey,” I continued. “How can you measure productivity based on lines of code or commits if multiple people – sometimes two, but often even more – have collaborated on the solution?”

Dumping kerosene

But most importantly, I forgot to add that there’s this thing called Goodhart’s Law, which can be freely interpreted as: when a measure becomes a target, it ceases to be a good measure.
A good example of Goodhart’s Law was presented to me by a friend during a recent conversation. At one point, the Royal Dutch Air Force aimed to ensure sufficient flight movements at air bases in practice and decided to measure the amount of kerosene being used. More kerosene indicated more movements, which suggested a more active air base. It sounds simple, but in practice, this measure led personnel at air bases to dump kerosene instead of flying more, resulting in highly polluted grounds.

In the case of developers, let’s assume that an organization would simply count the number of commits people make and consider more commits as indicative of greater productivity. Developers might then break their code into smaller parts and check them in to appear more productive. Although smaller commits are, in fact, more productive, counting them is still not a good measure. Developers will soon begin dumping kerosene. 

The question is: can (and should) you measure software delivery productivity, and can teams improve themselves using measures, instead of adhering to Goodhart’s Law?

Continued to read