At The AI Enterprise Conference 2026, leaders from Tokio Marine, Morgan Stanley and The Fedcap Group described what happens after the first agent ships. They covered the validation tax on human reviewers, why workflows fail more often than models, and why proprietary context is where advantage builds.
Key takeaways
- Deployed is not scaled. Most of the audience had an agent in production, but none ran more than 25% of their business on agents.
- Beware the validation tax. Gallic said checking every step makes humans the bottleneck and errors rise as fatigue sets in.
- Workflows fail more than models. Sadeghi said scaling breaks on enterprise systems and APIs that aren't AI-ready.
- Context is the advantage. Gallic argued automation gains will even out, while proprietary context is where returns accrue.
- Use agents to build workflows. Prototype an integration with agents, then persist it into a rigid, low-cost workflow.
Most enterprises now have an AI agent in production somewhere, but few run much of their business on agents. When moderator Steven Willmott asked the audience at The AI Enterprise Conference 2026 in New York who had an agent deployed, most hands went up. When he asked who ran more than 25% of their business on agents, none did.
Willmott, CEO of Safe Intelligence, led the panel "Agents in the Enterprise: Context, Control, and Reality" in a session sponsored by Safe Intelligence. He was joined by Patrick Gallic, Head of International AI Hub at Tokio Marine; Atena Sadeghi, Global Head of Enterprise Data Engineering & Analytics at The Fedcap Group; and Ken Zhang, Distinguished Engineer and Global Head of GenAI, Data Science & Engineering at Morgan Stanley. Willmott noted that panelists spoke for themselves, not their companies.
Where are AI agents creating business value?
Sadeghi said the agents that succeed start from business strategy, not from a desire to try a new model. Her team built a solution for employees who spent time searching many dashboards and systems for case information, a friction she linked to turnover. Developers wanted to build an AI assistant, she said, but "the AI assistant is not your solution." The question was what the organization gets in return.
Gallic said his company, which operates many acquired businesses under a federated model, uses agents in about 60 areas. They work best where output can be validated, such as code that passes tests or extraction accuracy that can be measured. In underwriting, agents do the research and preparation, and a human makes the final decision. A process that took three hours now takes six minutes, he said.
He also argued that "AI should be deployed at the core of your business. It cannot be deployed at the peripheral." For an insurer, that means claims and underwriting rather than an internal HR chatbot.
What is the validation tax?
Gallic distinguished agents from AI co-workers that a person checks at every step. When a human validates everything, he said, the human becomes the constraint in the workflow, fatigue sets in and errors rise over time. Real value comes from agents that carry out multi-step reasoning with their own tools, permissions and context.
Willmott drew a related line between a human in the loop, who helps create the output, and a human on the loop, who only reviews it. Reviewing the last 10% of an AI's work can be more frustrating than doing the whole job, he said, and he called the on-the-loop approach a very dangerous pattern because it satisfies a rule without real oversight.
Zhang said the level of review should follow the risk. A summarization agent carries less risk than one that emails clients, which should not act without permission. He described validating results after each step, using debugging tools, LLM-as-a-judge methods and human review.
Why do AI agents stall when enterprises try to scale them?
Gallic said the challenge has shifted from pilots to scaling one deployment across many business units, with security first among the issues. He compared agents to new hires: organizations first test whether they are competent, then cap the risk they can take on, the way every employee has a contract signing limit.
Sadeghi said the gap is rarely the model. Scaling means calling many enterprise systems and APIs that aren't AI-ready, and most of the time, she said, it is the workflow that fails, not the model. She also cited change management, since adoption can drop when employees fear agents will replace them.
Zhang said prototypes are often built on small or biased data sets with simplified features. Turning one into a product can require fuller data, orchestration, guardrails, retrieval, error control and sometimes splitting a single agent into several.
Where does enterprise context create an advantage?
Zhang said many agents need both short-term context in the prompt and long-term context from a well-designed retrieval system that combines semantic, keyword, metadata and graph search. Information can be lost in long prompts, weak retrieval or handoffs between agents, so he recommended validation at each component.
Sadeghi said context must respect role-based access, since a branch manager and an employee need different information from the same data. Teams also have to choose between a model's built-in memory and enterprise memory they control, and she said proprietary knowledge should stay in-house.
Gallic argued that productivity gains from workflow automation will even out across industries within a few years. "The context layer for your organization is where all your alpha is going to accrue," he said, meaning proprietary data and the know-how employees carry.
Should agents run workflows or design them?
Gallic said most workflows need generative AI for only a small share of steps. AI is most useful "as a non-deterministic solution to create a deterministic process," helping teams document and build processes, with agents reserved for open-ended problems. He described building a working email-to-podcast pipeline with coding agents in about four hours.
Willmott summed up the point: agents can work out an integration between two systems, but running it that way for a year costs tokens and may fail occasionally. Better to prototype with agents, then persist the result into a rigid workflow.
Meet enterprise AI leaders in New York
The AI Enterprise Conference returns Tuesday, August 31, 2027, at Pier Sixty in New York City. Registration is coming soon. Learn more on the conference page.
Frequently asked questions
What is the difference between human in the loop and human on the loop?
Willmott said a human in the loop helps create the output, while a human on the loop only reviews it, which he called a dangerous pattern because review can become a formality.
Why do AI agents fail to scale in enterprises?
Panelists said models are rarely the problem. Enterprise systems and APIs often aren't AI-ready, prototypes rely on small or biased data, and adoption suffers without change management.
Where should enterprises deploy AI agents first?
Gallic said at the core of the business, such as claims and underwriting for an insurer, and Sadeghi said only where the business impact is defined up front.
Should AI agents run business workflows directly?
Not always. Gallic and Willmott suggested using agents to design and prototype processes, then persisting them as deterministic workflows that are cheaper and more reliable to run.
Watch: Agents in the Enterprise: Context, Control, and Reality on YouTube
Event: The AI Enterprise Conference · More coverage: The AI Enterprise Conference insights
