In a Data Science Connect expert panel, leaders from Zenlytic, Nexla, Alation and Dataiku defined AI agents as chatbots with a feedback loop. They explained why context and memory are separate problems, how explainability builds trust and why task-specific agents scale better than one generalist.
Key takeaways
- An agent is a chatbot with a feedback loop. Janssen compared agents to a thermostat's closed-loop control.
- Today's agents are supervised. Desai put the industry at level one or two of autonomy, like supervised self-driving.
- Context and memory are different problems. Context management is improving; models that remember remain early, Janssen said.
- Reliability comes from explainability. No LLM will be 100% accurate, so users need to see what the agent did and where answers came from.
- Keep agents task-specific. Complexity rises quickly with each task, so panelists favored focused agents and careful orchestration.
The simplest way to understand an AI agent is a thermostat, Zenlytic CEO Ryan Janssen told attendees of a Data Science Connect expert panel webinar on planning, reasoning and contextual memory in agentic AI. He was joined by Amey Desai, Chief Technology Officer at Nexla; Jonathan Bruce, VP, Strategic Customer Advocacy, at Alation; and Christian Capodville, Senior Director, Content and Product Marketing, at Dataiku. Brian Mink, co-founder and president of Data Science Connect, moderated the session, which was made possible by Nexla, Dataiku, SAP, Galileo, Zenlytic, Alation and Informatica.
The panel covered how to define agents, why memory remains unsolved, how to earn user trust and how to keep agent systems from collapsing under their own complexity.
What is agentic AI, and how is it different from a chatbot?
Capodville defined agents as LLM-powered systems that can take actions autonomously. A retrieval-augmented chatbot answers a question. "Agents are about what happens after that answer," he said. Use cases range along a spectrum from simple chatbots to autonomous workflows, and complexity rises "surprisingly quickly" as customers move along it.
Janssen, a former electrical engineer, defined an agent through closed-loop feedback, the principle behind thermostats and airplane autopilots. An agent is what you get when "you take a chatbot and you add a feedback loop onto it so it can evaluate its work," he said, which unlocks goal-seeking behavior.
Desai compared today's agents to Tesla's supervised self-driving mode. "I think of agentic AI as more of LLM workflows that are supervised today," he said, putting the industry at level one or two with three more levels to go.
Bruce described agents as a way to get work done "in a highly contextual fashion," built on the data and context organizations have already organized in an enterprise data catalog.
How do you design agents that plan and execute complex tasks?
Start by defining the boundary of the problem, Desai said. Agents can't yet scale to any number of tasks, but they have moved from handling one to three tasks a year earlier to double digits now. The work is largely product management: identify the specific complex tasks, then design an agentic system to solve them.
Janssen said the industry is at a peak of hype, building agents without knowing what they will do. Bruce urged teams to identify the business metric they are trying to move. Capodville suggested anchoring agents in what employees do today. At a certain level of complexity, he said, "your agents need some of the same accesses that your real life employees need," which reveals the systems and permissions the agents will have to orchestrate across.
How should AI agents handle context and memory?
Janssen treated context and memory as two distinct problems. "Context is king right now," he said. Models are intelligent enough. What they need is the right information at the right time, and there is a race to master context management.
Memory is the next frontier, he said. He compared today's models to a superhuman friend who has read every book but can't remember your last conversation. "That friend is an amnesiac," he said. The fix will be models that retain and index information on their own.
Desai said building knowledge graphs from existing metadata "taps out very quickly." Nexla has seen better results from fine-tuning a model specifically to retrieve context and avoid outdated information, paired with whichever general LLM a customer uses, and from continual learning.
How can enterprises make agentic AI reliable and trustworthy?
Capodville said trust improves when teams treat agents as systems, not assistants: log decisions, monitor them and evaluate outcomes as models and context change. Users also need to see which source an answer came from and verify it themselves.
Janssen said reliability is really a question of explainability. "I don't think any LLM will ever be 100% accurate," he said, but no human is either. He cited the early elevator: riders complained it was slow, and the fix was not speed but floor indicators that showed progress.
Desai cautioned against relying on LLMs to judge LLMs and suggested connecting evaluation sets to an enterprise's existing QA teams, who already know what right and wrong look like. On real-time use cases, he said, "I would be very skeptical of letting LLMs run anything on critical workflows today."
How do you govern and scale agentic AI?
Janssen said early text-to-SQL agents gave "this giant random number generator" unfettered access to the warehouse. Routing queries through a semantic layer adds governance and separates the LLM from the data. Bruce added that data contracts and federated, data-mesh-style ownership help. Desai disagreed, saying semantic layers and data contracts may matter less as agents start communicating across companies, and that new privacy research will be needed.
On scale, Desai warned that architectures with 20 or 30 agents get slow and costly. His hot take: "don't use a framework." Janssen said complexity rises exponentially with each added task and recommended task-specific agents over one generalist. Capodville compared it to an organization: no one person does everything, so the challenge becomes orchestration. "Smooth is fast," he said.
Closing advice: Desai urged co-agency with humans in the loop; Bruce said "it's all about trust"; Janssen said to start small and ask whether ChatGPT alone can solve the problem; and Capodville said not to reinvent processes the organization has already solved.
More agentic AI discussions in Data Science Connect webinars
Data Science Connect's 2027 calendar includes 24 free live webinars across four series: the AI × Enterprise Panel Series, AI Demo Days, Agentic AI Showcases and the Plug & Play Series. Agentic AI Showcases focus on agent design and deployment. See the webinar schedule.
Frequently asked questions
What is the difference between an AI agent and a chatbot?
Capodville said a chatbot answers a question while an agent takes action after the answer; Janssen said an agent adds a feedback loop so the system can evaluate and adjust its own work.
Why don't AI agents have memory?
Janssen said current models can't retain past interactions unless that context is supplied again, and models that retain and index information on their own are still in early research.
How do you make AI agents trustworthy?
Panelists recommended logging and monitoring decisions, showing users sources they can verify, explaining what the LLM is doing and using existing QA teams to evaluate outputs.
How many agents should an enterprise system use?
Desai warned that architectures with 20 or 30 agents get slow and expensive; the panel recommended keeping systems simple and agents focused on specific tasks.
Watch: Building Smarter Enterprises with Agentic AI: Planning, Reasoning, and Contextual Memory on YouTube
Event: Webinars · More coverage: Webinars insights
