Where Human Judgment Belongs In AI Workflows

Emily Lewis-Pinnell, Driving AI Adoption at Evaila.

gettyPeople are trusting AI with work that matters. That’s a sign of real progress in AI adoption, and it raises the bar for how organizations prepare people to use it well.

Stanford researchers analyzed 249,834 consumer conversations with Anthropic’s Claude. Among those containing an actionable task, 56% involved consequential or high-stakes work, with legal and financial guidance ranking highest. The study, accepted by Nature Human Behavior, also found that people engaged more actively as the stakes rose.​

For leaders, the implication is that AI fluency must include knowing when to question an answer, verify a claim or bring another person into the decision. Organizations need to build those capabilities into the workflow.

Much of the discussion about AI adoption focuses on access and usage. The Stanford research offers a different view: what people do once they’re in a conversation with AI.

Average conversation length rose from 6.5 turns for ephemeral tasks to 12.4 for high-stakes work. Human-led collaboration accounted for 72% of conversations. Significantly adapting AI output was the most common engagement pattern, reaching 60% for consequential tasks.

These numbers suggest that people often recognize when work requires more attention. They clarify the request, challenge the response and refine what AI produces. They remain involved in shaping the work.

The researchers also examined friction: conversational events that slow or impede progress. It appeared in roughly half of conversations and could lead to clarification, refinement or learning. Users attempted to recover in 78.7% of conversations containing friction, although recovery strategies varied in effectiveness.

Keep in mind that these are model-inferred patterns from one provider’s consumer accounts, not a representative sample of enterprise workflows. The researchers couldn’t see what happened after the chat ended. Longer conversations and more revisions don’t, by themselves, establish better outcomes.

Still, the findings challenge a narrow view of enablement. Teaching someone to get an answer is only part of the work. People also need practice evaluating whether that answer is useful, supported and appropriate to act on.

In our work with leaders at Evaila, we’ve seen people move beyond basic drafting as they gain confidence and tackle more demanding work. That progress requires practice evaluating and refining what AI produces.

That raises an operating question: If active engagement matters, how do organizations preserve it while capturing the efficiency AI offers?

Stanford’s Digital Economy Lab explored a related question in “The Enterprise AI Playbook.” In its sample of successful deployments, escalation-based models—where humans review exceptions—were associated with median productivity gains of 71%, compared with 30% for models requiring approval on every output.

Task selection helps explain the difference. Escalation models typically handle high-volume, recoverable work. Approval and collaboration models often serve regulated or high-stakes decisions. The comparison doesn’t establish that removing approvals causes better results.

Read together, the studies support a clear principle: Match human involvement to the consequences of the work. Routine, recoverable work can be a good candidate for exception handling once performance is established. Consequential decisions require explicit choices about who reviews the work, what they verify and when approval is needed.

Conversational friction and formal approvals serve different purposes. Questioning an answer helps develop and test it. An approval establishes who has authority to move it forward. Workflow design needs to account for both.

In my own work, I use a deliberate human-AI-human-AI cadence, with points where my judgment reenters as the work develops. I think of this as intentional friction. The purpose is to apply attention where it improves the result.

As agents take on more work, those decisions become more consequential. An output can trigger an action, update a record or reach a customer. Leaders need to define what the system can do, when a person must intervene and who owns the outcome.

1. Start with the workflow and its consequences. Choose a priority workflow and map where AI contributes today. Identify who relies on the output, what happens if it’s wrong and how easily an action can be reversed. Assign a business owner to decide where review, verification and escalation belong.

2. Teach evaluation alongside prompting. Give people a deliberately flawed output from work they understand. Ask them to identify unsupported claims, missing context and incorrect assumptions before revising it. Practice checking sources and calculations. A convincing explanation from AI isn’t sufficient verification.

3. Make decision rights explicit. Define what AI can complete independently, what requires approval and what triggers escalation. Give employees a practical route to someone qualified to resolve an issue. Review existing checkpoints against risk, required controls and actual performance, then simplify where the evidence supports it.

4. Measure the outcome and investigate the interaction. Track quality, errors, cycle time and the business result the workflow is meant to improve. Use rework, verification and escalation patterns to help explain performance. More interaction may reflect careful judgment, unclear instructions or a system that repeatedly fails. The outcome tells you which questions to ask.

Employees also need to know that challenging AI is expected. If teams are rewarded only for speed and usage, careful review can start to look like resistance. Managers shape that behavior through what they ask about, what they recognize and how they respond when someone raises a concern.

In my recent article on how agents shift AI adoption from broad enablement toward focused workflow redesign, I argued that the two reinforce each other. Broad enablement builds the confidence to question and evaluate. Work on a specific workflow gives that capability a purpose, an owner and a measurable result.

The next stage of AI adoption requires organizations to improve speed and judgment together. That takes deliberate choices about where people contribute, what they need to know and how success is assessed.

Where in your most important workflows does human judgment need to enter as AI takes on more of the work?

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?