Enabling the AI Data Scientist: Why Context Layers Need Training

AI agents already have general intelligence. What they lack, according to Rohan Kodialam, Co-Founder and CEO of Sphinx AI, is specific intelligence about how a particular business works, and the human effort needed to supply and maintain it is too expensive to scale. Kodialam spoke at the ALIGN AI Executive Summit New York City 2026 on June 25, in a session sponsored by Sphinx AI titled Enabling the AI Data Scientist: Turning Data Context into a Competitive Advantage.

Why don't data teams trust their agents?

Kodialam said he previously ran a data science and AI team at Citadel, where agents were used early to make data-driven decisions. The bottleneck quickly stopped being what agents could do. They kept misinterpreting processes and hallucinating, so the most useful thing a team member could do was "babysit": correct how a metric was defined, which process to use and how a result should be reported.

Many vendors now offer a memory, context or semantic layer where a company can describe how its business operates. Kodialam compared it to being handed a box and told that good things will happen once the business is inside it. Filling that box, he said, "is highly nontrivial."

Why isn't connecting agents to more data enough?

Kodialam described a pattern he sees in analytics teams. Connecting an agent directly to data produces low accuracy on routine BI questions. Giving it search access to dashboards and code helps little, because the problem is synthesis, not retrieval. Having engineers and data scientists write down what every table and metric means does work, but the knowledge decays quickly as data drifts, and keeping it current takes continuous manual effort.

"It's like an unendurably expensive process to have humans populating your semantic layer," he said. By the time a team finishes a second use case, the first has drifted.

How does Sphinx build the context layer?

Sphinx trains on artifacts a company already has, including notebooks, dbt code, ETL pipelines, BI dashboards and even executive decks, and produces a graph-like knowledge structure any agent can use. Kodialam described the approach as similar to gradient descent: the system tries to redo past work, such as a revenue forecast, finds where it got things wrong, like using a bronze table instead of a gold one, and learns from the mistakes. Continual learning runs on the same inputs, for example through daily pulls from GitHub, SharePoint, Google Drive or Notion.

Beyond accuracy, he said the right knowledge layer improves consistency, auditability and cost, since agents stop searching every tool and data source for an answer. He said costs on some analytical tasks fell by as much as 90 percent.

Where do humans fit in?

Most enterprises have conflicting definitions of the same metric. Kodialam mentioned a pilot team with about five definitions of revenue. Instead of averaging out conflicting signals, the system flags these low-confidence areas and turns them into targeted questions for subject matter experts. Rather than babysitting for months, he said, experts might spend a weekend answering 30 or 40 questions.

Because the knowledge is stored in plain language, people can read, approve, edit and permission it. Kodialam listed five governance benefits: access control, a governed knowledge lifecycle, auditability, collaboration among many agents and global fixes. When an answer is wrong, a team can trace it to a specific definition, see who approved it and fix it everywhere.

What does the rollout look like?

Kodialam outlined a general roadmap, with or without Sphinx:

• Knowledge transfer: give the system existing artifacts and let it build a baseline understanding, which he said reaches about 95 percent correctness in most cases.
• Iterative improvement: have experts resolve flagged conflicts over a week or two, moving toward 99 percent, which also cleans up definitions across the organization.
• Evaluation: test accuracy, repeatability and cost. "Ask the same question twice, get the same answer," he said.
• Onboarding: plug the layer into any agent, whether Claude, GPT or an in-house system.

He said onboarding typically takes three to five weeks rather than six to 18 months.

Where to hear more in 2027

Data and AI leaders will keep comparing notes on context and agents at the 2027 ALIGN AI Executive Summits, invitation-only gatherings in eight U.S. cities: Atlanta, New York City, Dallas, Chicago, San Francisco, Boston, DC and Los Angeles. Registration is coming soon. See the ALIGN AI Executive Summits page.

Frequently asked questions

What is a context or semantic layer for AI?

Infrastructure that captures how a business operates, such as metric definitions and data flows, so agents can use that knowledge.

Why do manual semantic layers fail?

Kodialam said they are costly to build and decay as data drifts, so they need continuous human upkeep.

How should humans be involved?

By reviewing targeted questions about low-confidence areas, such as conflicting metric definitions, rather than supervising agents continuously.

This article is based on remarks by Rohan Kodialam, Co-Founder & CEO, Sphinx AI at ALIGN AI Executive Summit New York City 2026 on June 25, 2026. More coverage: ALIGN AI Executive Summits.

Filed under: Data & AI Infrastructure, Agentic AI, AI Governance & Trust