Why Engineering Productivity Metrics Keep Misleading CTOs

Elton Chan is the Co-Founder of Second Talent, a solution that connects global tech leaders with AI-native engineering talent across Asia.

getty​Engineering teams have more ways to measure developer activity than ever. CTOs can track pull requests, commits, deployment frequency, review times and AI tool usage.

But a busy dashboard can still leave one question unanswered: Is the team delivering better software, or producing more work that someone else has to process?​

I’ve seen how AI coding tools can help engineers produce a first version faster, but the time saved during implementation can get absorbed by code reviews, test updates and integration issues. Even if the team appears more productive when measured by code output, the time it takes to move a change from development to a reliable production release often hasn’t improved.

To solve this, engineering leaders need to measure the entire delivery process, not just the speed of individual tasks.

​A team can write more code and still ship features at the same pace. Developers might open twice as many pull requests, only for reviewers to spend more time checking them. More code enters the pipeline, but the team still has to test it, integrate it and deal with whatever breaks.

The 2025 DORA report found AI adoption is associated with higher software delivery throughput, while also finding a negative relationship with delivery stability.

For a CTO, the practical question is what happens between writing code and getting a change safely into production. If implementation speeds up but testing, review or integration falls behind, the team may be generating work faster without delivering proportionally more value.

Lines of code, commit counts and tickets completed are easy to count. They are also easy to misread.

A developer who splits one change into 10 commits can look more active than someone who delivers the same change in two. A team can close more tickets by breaking work into smaller pieces without improving its delivery rate.

The SPACE framework offers a broader approach, looking at satisfaction and well-being, performance, activity, communication and collaboration, and efficiency and flow. Its central point is straightforward: No single metric captures developer productivity.

CTOs do not need to measure everything in the framework, but they should avoid drawing conclusions from one number in isolation. A rise in completed tickets means little without knowing whether the work reached users, met quality expectations and reduced the problems the team was hired to solve.

A pull request waiting three days for review is a different problem from a feature waiting three days for a product decision. A test environment that fails unpredictably creates a different delay from code that repeatedly needs rework.

Yet a dashboard showing that delivery has slowed down may not tell a CTO which of these problems is responsible.

Look at where work waits and where it has to be repeated. Review wait times can expose a shortage of reviewer capacity, while rework and failed changes can point to testing or quality problems. Long lead times between a task starting and reaching production can reveal delays across several teams.

The response should follow the evidence. Hiring more engineers will not clear a review queue if nobody has time to review. Adding another testing tool will not fix a team that receives changing requirements halfway through implementation.

AI tools can shorten implementation time, but that saving needs to be assessed alongside the work that follows. If developers spend less time writing code and more time checking generated output, resolving integration problems or fixing regressions, measuring coding speed alone gives CTOs an incomplete picture.

The 2025 DORA research finding that AI adoption is associated with higher throughput but also with lower delivery stability offers a useful starting point. Leaders evaluating their own AI investment need to examine both sides rather than relying on adoption rates or developers’ reported time savings.

To do so, compare delivery before and after introducing a tool. Look at how long changes take to reach production, how often they cause problems, how much rework they generate and whether the team is delivering outcomes that users need.

Then, ask engineers where the time saved has gone. Their answers can help explain changes in the metrics.

If implementation time falls while review queues and rework grow, the next investment may need to be in testing, review capacity or the development workflow rather than another AI tool.

A useful engineering dashboard should help leaders understand how work moves from a business need to a reliable release. A small set of measures can cover delivery speed, change failure rate, review delays, rework and the outcome the work was meant to achieve.

These measures should be read together and at the level where teams can act on them. They are useful for finding constraints and deciding where to invest. They are much less useful as a league table ranking individual engineers by commits, tickets or lines of code.

Instead of trying to make a dashboard look more productive, understand whether the engineering organization is delivering better software, where work is getting stuck and what needs to change.​​

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?