After The Cheap-Token Era: Three Levers For Efficient, ROI-Positive AI Use
CTO at Deque Systems and author of the “Agile Accessibility Handbook, A Practical Guide to Accessible Software Development at Scale.”
gettyThe cheap-token era is over, and with it, the assumption that more AI automatically means better results.
Organizations that get the most from AI investment will be the ones that develop the discipline to use it deliberately. That means three things: a reliable way to measure what you’re spending against what you’re getting; a way to ensure the speed of adoption isn’t generating technical debt that undermines the value you’re creating; and a principled process for deciding when to use AI and when a deterministic, rules-based approach will serve you better.
Fortunately, a practical framework for all three exists. In part one of this series, I explored what digital accessibility has taught us about using AI effectively—specifically, that AI and deterministic tools each excel at different categories of task, and that knowing which is which is what separates efficient AI investment from an accelerating bill. Here, I’ll share three practical levers for putting that into practice.
Every token you send costs money. A naive implementation that loads everything available into the context window—for example, the full pull request diff or every potentially relevant piece of information—will be expensive and often ineffective. An agent given a focused context and the right tools to search and investigate what it needs will frequently outperform and be more cost-effective.
The right approach is to give AI only what it needs, not everything you have. In a pull request review process, for example, giving the agent tools to investigate changes on its own often produces better results than preloading the entire diff. That single discipline can reduce token consumption dramatically without reducing output quality.
Every major AI provider maintains a cache of recent context. If a subsequent call reuses that cached context, it costs significantly less than rebuilding it from scratch. A well-designed caching strategy—for example, one that sequences calls to maximize cache hit rate rather than parallelizing everything—can reduce costs to a fraction of a naive implementation for the same outcome.
This requires understanding your cache hit rate, your turn structure and how your calls are sequenced, whether you’re using a prebuilt agent or building directly on an API. But the effort is worth it, because the savings are substantial.
Not every task needs AI. Some tasks are faster, cheaper and more consistent when handled by rules-based, deterministic tools that produce the same result every time, with no tokens required.
Before deploying AI on any workflow, break it into subtasks and determine which ones actually require AI judgment. A pull request review, for instance, might include syntax and style checking, security vulnerability scanning and accessibility conformance checking—tasks that deterministic tools handle thoroughly, consistently and cheaply—alongside architectural review and plain-language explanation of changes, where AI adds genuine value.
Assign each category of task to the approach best suited to it. That can drive costs down, velocity up and consistency throughout.
All three levers require systematic capture of metrics across the full development lifecycle. This can include everything from capturing cost-per-call and cache hit rates to turn counts and context size. The goal is that when something changes, you can explain not just what happened, but why.
Session logs are particularly valuable: When a workflow becomes unexpectedly slow or expensive, those logs give you the data to diagnose the issues. AI can help you analyze them to identify what drove the change.
It’s essential to build this measurement infrastructure before you need it. That way, you’re not just controlling costs; you’re building the analytical foundation that lets you move faster with confidence, make better investment decisions and clearly demonstrate what your AI spend is actually delivering.
AI is an extraordinary tool, but it’s potentially a very expensive one. The organizations that get the most from their investment will be the ones with the discipline to use the tool appropriately.
Optimize your context. Use caching strategically. Do deterministic tasks deterministically. Measure cost, speed and quality from day one. That discipline is what separates the organizations that lead in the token era from those still trying to figure out where the money went.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
