Learn effective strategies to measure and improve developer productivity. Discover valuable tips for optimizing performance and achieving project success.
Technology has found its way into every facet of the modern workplace. Operating costs, security, communications, employee satisfaction and the customer base are all part of the technology equation. A savvy CIO knows there is a direct correlation between a high-performing IT organization and a strong-performing business.
SEE: Gain Agile and Scrum skills to help boost your productivity.
As a technology leader, you need to advocate for the ability to measure how fast your team is going and that they are headed in the right direction. You can’t improve what you don’t measure.
Jump to:
Trying to gauge how a technical team is delivering can be tricky. A team is a collection of individuals. And in the case of an IT organization, those individuals are performing discrete, complex tasks. Over the years, managers of software development teams have tried a lot of approaches to measuring productivity, the majority of which suffer from two fundamental flaws:
These flawed approaches have given rise to a handful of anti-patterns that not only fail to provide meaningful productivity metrics but can lead to poor team morale.
Perhaps the most famous, and most hated, failed attempt to measure developer productivity is counting lines of code. There is little correlation between how many lines of code a developer writes and the overall value that developer is delivering to the organization.
In fact, rewarding developers for writing lines of code results in code bloat and ultimately incurs higher sustainment costs.
With the prevalence of Agile in software development, at some point, some Agile coaches will likely recommend using velocity as a way to measure your team’s productivity. Team velocity, not individual contributor velocity, is a useful metric for planning workloads.
As a measure of productivity, however, it falls short. Equating velocity to productivity will only cause developers to inflate estimates thereby not only misrepresenting the team’s effectiveness but potentially invalidating the measure’s usefulness in capacity planning.
In many consulting organizations, a developer’s utilization — i.e. how much time they spend working on the code — is used as a proxy for productivity. This is doubly flawed because we all know effort does not always mean results and because this measure incentivizes project managers to keep developers 100% utilized.
In mathematics, queue theory tells us that as utilization reaches 100%, lead times approach infinity. That’s because a 100% utilized resource has no capacity for innovation, improvement or change.
In 2018, Nicole Forsgren, Jez Humble and Gene Kim published Accelerate, which included cluster analysis of over 23,000 responses from more than 2,000 unique organizations. They found four common characteristics in the data that helped to categorize software development teams as high performers, medium performers or low performers:
| Measure | High performers | Mid performers | Low performers |
|---|---|---|---|
| Lead time for changes | Less than one hour | Between one week and one month | Between one week and one month |
| Development frequency | On demand (multiple times per day) | Between once per week and once per month | Between once per week and once per month |
| Mean time to recover | Less than one hour | Less than one day | Between one day and one week |
| Change failure rate | 0–15% | 0–15% | 31–45% |
Table source: Accelerate, published by IT Revolution 2018.
Besides strictly code-based measures, there are several cultural factors that can help gauge how your software team is performing.
Once you know what measures indicate how the team is performing, as a CIO you then have to set aside the time and resources to build a dashboard to measure. It’s likely, the data required will not come from a single place, so you’ll need to capture and transform the data from multiple sources and then use a custom visualization tool like Tableau or PowerBI to present it.
It’s better to start simple and expand where you are getting the most value. Frequently, you can get most of the quantitative data you need from APIs on your version control system and code pipelines. For the more qualitative measures, consider using quarterly surveys.
At the end of the day, collecting data and metrics, even just a handful, is wasted effort if the organization is not continuously reviewing the data and using it to make course corrections.
While you may get some pockets of excellence if you leave it to individual teams, setting aside time as an organization to regularly review the metrics, gather insights and drive data-informed change is the quickest route to becoming a high-performing IT enterprise.
