The Next Software Revolution After AI Is Efficiency
Itamar Syn-Hershko, Founder of NeverBlink AI and BigData Boutique, helps companies confidently operate and optimize data platforms at scale.
gettyFor decades, we’ve made software easier to write, even when it meant making it more expensive to run.
We introduced programming languages, frameworks and layers of abstraction that allowed developers to move faster. Along the way, we accepted higher memory consumption, more CPU usage and increasingly complex infrastructure.
For the most part, this made sense. Developers were expensive, hardware kept getting cheaper and getting products to market quickly was often more important than squeezing the most out of every server.
I think we’re approaching a point where that trade-off no longer makes sense.
AI is changing the cost of writing software, while the growing demand for computing resources is making efficiency more important than it has been in years. Together, these forces could change not just how we build software but also the economics of the entire software industry.
Take Java, for example. Much of its success came from making developers more productive. Automatic memory management, a rich ecosystem and portability made it easier to build and maintain complex applications without worrying about many of the details involved in lower-level programming.
The trade-off was additional runtime overhead. Garbage collection, managed memory and abstraction layers can require considerably more resources than a carefully optimized native implementation.
This was often a perfectly reasonable compromise. If using Java saved months of engineering work, spending a little more on servers was easy to justify.
But what happens when AI can write, maintain and optimize complex C++ or Rust code at a fraction of the previous engineering cost? We should start questioning whether some of the abstractions we’ve relied on for decades are still worth their price.
I’m not suggesting we rewrite every Java application in assembly. There are good reasons to use managed languages, including their mature tooling, ecosystems and safety characteristics. AI-generated code also introduces its own risks and still needs careful review.
But the argument that lower-level programming is too expensive or complicated to justify is becoming much weaker.
At the same time, the infrastructure side of the equation is changing.
AI is creating enormous demand for computing capacity, memory, electricity and data center space. The International Energy Agency projects that global data center electricity consumption will roughly double by 2030.
We cannot keep solving every software performance problem by adding more hardware. At some point, the availability and cost of that hardware become serious constraints.
There’s also growing pressure to make software cheaper.
AI is already reducing the time and effort involved in producing software, and we should expect some of those savings to reach customers. As it becomes cheaper to build competing products, companies will find it harder to justify high prices based on development costs alone.
This puts infrastructure efficiency at the center of the conversation. If two products offer comparable functionality but one requires a fraction of the hardware to operate, the difference can be substantial. The more efficient product can offer lower prices, better margins or both.
Search engines and databases are good places to look for early examples.
Traditional search architectures (Lucene-based especially, such as Elasticsearch and OpenSearch) often require continuously provisioned compute resources and substantial memory to maintain indexes and deliver low-latency queries. This can make infrastructure costs difficult to align with fluctuating workloads.
Companies such as turbopuffer and WarpSearch are challenging those assumptions. Both use an object-storage-first architecture for vector and full-text search, reducing the need to keep large indexes on expensive infrastructure. A Google Cloud case study reports search and storage cost reductions of 90% for its approach.
There are examples outside search as well, including ClickHouse, Neon, Bun and more.
These products aren’t necessarily the result of AI-assisted development. Many of their innovations started before today’s coding agents became widely available.
What interests me is what happens when the engineering effort required to build systems like these drops dramatically.
For engineering leaders, this should change how we evaluate technology choices.
We spend a lot of time measuring developer productivity, feature delivery and time to market. We should be paying just as much attention to what our software costs to operate.
That starts with measuring actual resource consumption. Cost per query, cost per transaction, memory per workload and CPU utilization are not just infrastructure metrics. They directly affect the economics of the product.
It also means questioning architectural decisions we’ve come to accept. Does that service really need to run continuously? Does that dataset need to stay in memory? Are we using the right algorithms and data structures? Could a more efficient implementation eliminate entire servers?
And perhaps most importantly, we should be using AI for more than producing code faster.
AI can help investigate performance bottlenecks, evaluate alternative algorithms, rewrite expensive components and explore lower-level implementations that would previously have required too much engineering effort.
There are still trade-offs. Reliability, security, correctness and maintainability matter. An implementation that saves 20% in CPU costs but introduces serious operational risks is hardly an improvement.
The point is that we now have more ways to pursue efficiency without paying the same development cost we once did.
I expect we’ll see more companies challenge established software categories by building products that achieve similar results with much less infrastructure.
Some will use better algorithms. Others will rethink storage architectures, eliminate unnecessary compute or replace expensive runtime components. AI will make many of these approaches easier to implement.
For established software vendors, this creates a real challenge. Their existing architectures may have been designed around assumptions that no longer hold, while newer competitors have the freedom to start from scratch.
For customers, it should eventually mean more capable software at lower prices.
We’ve spent decades making software easier and cheaper to write and accepting the infrastructure overhead that came with it.
Now that writing software is becoming cheaper, it’s time to focus on making it cheaper to run.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?