AI May Write The Code, But Real Enterprise Value Is In Understanding It
Rob Kim, CTO at Presidio, driving innovation and digital transformation through AI, cloud and next-gen infrastructure solutions.
gettyWe’re watching a dynamic play out with AI at enterprise scale. Organizations that race ahead with AI agent deployments are discovering an uncomfortable truth: velocity without understanding isn’t transformation—it’s accumulation (AI slop). Every AI workflow shipped without a clear mental model of what it does, who owns it or how it evolves adds to a balance sheet nobody is tracking.
The following phrase from Notion design engineer Jeffrey Litt has reshaped how I think about both coding and corporate AI strategy.
“Understanding isn’t a verification step. It’s the substrate for participation.”
This reframes the entire AI conversation—both on the keyboard and in the boardroom. Verification is being automated (so stop trying to impede). What can’t be automated is the human capacity to generate new ideas, weigh tradeoffs and steer direction, and that capacity depends entirely on whether we understand the systems we’re building with (namely AI output).
In IT, we talk about technical debt constantly but rarely talk about cognitive debt—the slow erosion of a team’s mental model as code accumulates faster than our ability to comprehend it.
When developers can’t reason about their own systems, they stop being authors and become passengers. It’s the essence of vibe coding; shipping changes that work, but for reasons nobody on the team can articulate. The impact may not show up until several months after the first sprint, with a critical bug that surfaces without anyone knowing where it came from.
The immediate response is to gate everything behind review—in the form of human approval. That’s a losing strategy in the AI era because agents outproduce human reviewers by exponentially. The more productive (but challenging) response is to redesign how teams learn what their tools have built.
This worked when humans wrote every line of code, because the approver generally understood what they were approving.
In the AI era, that assumption breaks. Approving output you cannot evaluate is not governance, but theater. Understanding is the only form of participation that scales: when a team genuinely comprehends what their agents are building, they won’t just “rubber-stamp it,” but instead they reshape it, steer it and teach the system what good looks like.
The human element doesn’t act as a gate of control (and slow things down) but instead is the force that guides a more accurate output—zeroshot. This approach can be simplified with the following three techniques (at least to start):
A traditional PR shows you what changed but a curriculum-style explanation shows you why it changed, what system it lives in and what mental model you need before the diff makes sense.
Picture a change that converts a 2D top-down game into an isometric view. A flat diff is unreadable. A curriculum-style explainer starts with the rendering engine, then the coordinate system, then a plain-English summary (“we’re faking 3D with 2D drawing tricks”), then an interactive widget where you drag objects around to see the effect—only then the annotated line-by-line change.
Litt built a tool that generates these documents automatically and drops them into his agentic software as collaborative pages. Quizzes at the end of each section force active recall—a technique borrowed straight from educational research. You can’t merge until you pass.
The unlock is to treat explanations as a first-class artifact, not a comment thread.
This pattern borrows from Seymour Papert, the MIT researcher who argued decades ago that children learn math best by “living in Mathland.” Now, just replace “Mathland” with the codebase, and you get the concept of microworlds: interactive simulations where developers explore code behavior rather than reading about it.
A prolog interpreter debugger lets you step through state changes manually, but a clickable simulation allows you to walk through migrating a website between frameworks, converting to playable environments vs. docs.
Microworlds collapse the gap between reading code and running code. They turn comprehension from a passive activity into an interactive one.
Most software is built by teams, but most AI-coding workflows are deeply individual.
Litt’s team inverted this. AI agents live in shared chat channels, and the explanations live in shared documents. Additionally, the coding assistants are integrated into team workspaces versus living in individual IDEs, resulting in a collective cognitive map where everyone sees the same model of the system. This allows “disagreements” to surface early instead of in production. We have applied this procedurally across our own development teams.
This isn’t an AI challenge as the technology already exists. The real challenge is shifting cultural where treating “understanding” as a team deliverable, not a personal one.
“The computer is a tool to help you do what you want to do, not to tell you what you should do.”
In essence, “level humans up.” AI doesn’t have to erode that, rather, it can scaffold it. Don’t automate away understanding in trade for manual (human) control. The temptation in any productivity revolution is to mistake output for progress—heck, we did it with tokens (remember tokenmaxxing?). We can ship lines of code, merge PRs and close tickets infinitely faster with AI, but none of it matters if the team that built it can’t explain it six months from now.
The teams (and enterprises) that win the next decade will be those who treat AI as a learning partner versus a replacement for thought.
Cognitive debt compounds, while understanding compounds faster—if you willingly invest in it.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?