Stop Measuring Technology Adoption And Start Measuring What It Changes

Gayathri Kolandaisami | Doctoral Researcher | Human-centered digital and AI transformation in water utilities and critical infrastructure.

gettyA technology investment can succeed on paper but still fail to solve the problem it was meant to solve.

I saw this happen long before the AI era. When I started my technology career in 2005, my job was to build and maintain the software used by my clients. Over time, I learned that building the technology was only part of the equation. We also needed to understand why it was needed and how the work was being done:

1. What problem were we trying to solve, or what improvement did we want in productivity, efficiency or service delivery?

2. How did the current process work?

3. What should the ideal process look like?

That way of thinking stayed with me as my career moved from software development into larger digital transformation projects. It left me with a principle I still believe today: Technology needs to work for us. We don’t work for the technology.

I was in a position to observe an organization’s reaction when an AI coding assistant was introduced to its developers. The management had invested in this technology long before I got involved in the organization. However, they hadn’t understood or researched where and how that technology could help the developers, what the benefits of using it would be or whether the tool was apt for their environment. The outcome was a KPI introduced at an organizational level to measure the tool’s adoption. So, a developer was considered to have met this part of their KPI by simply showing that they’d adopted this tool.

This left me asking myself: Had we moved away from measuring the outcomes of the technology to simply measuring its adoption? That’s when I realized that we’d started working for the technology and not the other way around.

Make no mistake—every cell in my body screams technology. I want to develop new technologies that could solve a lot of global challenges. I’m a strong believer in new technologies, and I’m fascinated by how quickly AI is evolving. But I also think that we’re forgetting why we invented these technologies in the first place. Sometimes, we race toward adopting the newest technology without knowing what we were trying to make better.

Technology is a means to solve a problem, improve a process or create a better outcome. The technology itself isn’t the outcome.

I came across a 2025 study published in Management Science. The researchers estimated that using an AI coding assistant increased completed tasks by 26.08%. In contrast, another 2025 study that examined experienced open-source developers working on mature repositories found that developers took 19% longer when using the AI tools that were tested.

The lesson from these studies is that technology doesn’t produce the same result in different environments.​

Organizations can easily measure the wrong outcome.

Take software development, for example. Building new software or making a change to existing software can involve requirements gathering, design, programming, testing, bug fixing and implementation. Let’s say we implemented an AI coding assistant. We may think that this tool could cut programming time substantially. However, that isn’t the whole picture, is it? What happens if the AI-assisted code requires more testing? What if developers spend additional time reviewing and correcting it? What if defects increase, and some of the time saved in programming is consumed later in the process?

If we measure only programming time in this scenario, we may conclude that the technology has been highly successful. However, if we measure the entire journey from requirements to delivery, we may reach a very different conclusion.

The entire process matters when measuring the outcome of the technology. If the problem that the technology is addressing is slow delivery, measure the complete cycle from requirements to release. If the problem is quality, understand defects and rework. If the objective is greater capacity, determine whether the team can actually complete more meaningful work.

The metric should originate from the problem, the context and the process, not simply from the technology.

I prefer piloting the technology before widely adopting it. A pilot isn’t simply an opportunity to prove that a technology works. It’s an opportunity to discover whether it works for your people, your processes and your particular environment.

If employees aren’t using a technology, that should initially be treated as information rather than a performance problem. Perhaps they need training. Perhaps the technology doesn’t fit their workflow. Perhaps it works well for some tasks but not others. Perhaps the existing technology environment, including legacy systems or code, limits its usefulness. Or perhaps it simply isn’t solving a sufficiently important problem.

Adoption is only one part of the story. Technology can also affect employee and customer experience. It may remove repetitive work and allow employees to focus on more meaningful activities, but it may also create additional work when people have to check, correct or validate its output. Some outcomes are tangible, such as time saved, costs reduced, greater capacity or more work completed. Others are harder to quantify, such as employee satisfaction, customer satisfaction, trust and confidence in the service being delivered. They’re still outcomes.

As organizations embrace increasingly powerful technologies, the temptation will be to measure what’s easiest to see: licenses purchased, systems deployed, users onboarded or usage rates. Those numbers have their place, but they don’t answer the most important question: What became better because we introduced the technology?

Always remember: Technology needs to work for us. We don’t work for the technology.

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?