In a Data Science Connect expert panel, leaders from AT&T, SAP, Posit, Revefi and Rubrik discussed how to give AI agents autonomy without losing control. They covered policy layers outside the model, risk-tiered approvals, agent identities, end-to-end tracing and why humans make poor review machines.
Key takeaways
- Governance moves to runtime. Trivedi said that once agents plan their own steps, policy has to be enforced as they act, not only at design time.
- Keep policy outside the model. Least-privilege permissions per agent and a policy layer separate from the agent's reasoning stop bad actions when judgment fails.
- Match oversight to risk. Irreversible actions and those involving money, personal data or external communication need human approval; low-risk ones can run freely.
- Trace why, not just what. Panelists described agent identities with scoped tokens and end-to-end tracing that works like a flight recorder.
- Humans make poor review machines. Altman said people often approve agent output without reviewing it, so systems must give them enough context to decide.
As AI agents move from demos into enterprise workflows, the hard question is no longer whether they can act on their own but how organizations stay in control when they do. Five practitioners took up that question in the Data Science Connect expert panel webinar "Operationalizing Agentic AI: Governance, Observability, and Human Control at Scale."
Dr. Frances Boykin, Director of Advanced Analytics at AT&T, moderated. The panelists were Shubham Trivedi, Digital Solution Advisor at SAP; Sara Altman, Senior Developer Advocate at Posit; Girish Bhat, SVP at Revefi; and Paul St Vil, AI CTO at Rubrik. Boykin noted that panelists' comments did not reflect the views of their organizations.
What makes an AI system truly agentic?
Bhat described automation as a power tool and an agent as a teammate. Automation executes predetermined pipelines with few unknowns, he said, while an agent can follow a plan, adjust it and make judgment calls, which makes it non-deterministic.
Trivedi said agents plan their own steps, invoke tools mid-execution and carry memory across sessions. Once an agent decides to act, he said, governance stops being a design-time conversation and becomes a runtime one. Altman called it a shift from an instruction-directed system to a goal-directed one.
What does a practical governance framework for AI agents look like?
Trivedi outlined three layers: tight scoping, least-privilege permissions for each agent rather than a shared service account, and "the policy layer should sit outside the agent's own reasoning," so a bad action is stopped even when the model's judgment fails. He added tiered approvals. Low-risk actions can run freely, but anything irreversible, involving personal data, money or external communication needs a human.
That mirrors how organizations already work, he said: a finance analyst can approve spending only up to a limit, and anything above it goes to a manager. He also suggested agents check each action against a written policy document, which makes their decisions auditable by default.
For guardrails that adapt as agents reason, St Vil favored using AI to govern AI, such as LLM-as-a-judge approaches that watch an agent's intent, alongside deterministic controls like data loss prevention. Bhat said Revefi asks its agent to list the steps it plans to take, along with their cost implications, before it acts.
How should teams observe multi-agent systems?
Asked about tracing decisions and handoffs between agents using MCP, St Vil pointed to agent identities: an agent acting for a human receives a scoped token with restricted privileges for a limited time. Observability without control, he said, isn't worth it. "You're trying to observe so that you can restrict and scope."
Bhat described end-to-end tracing that covers who sent a prompt, whether the requester was an agent or a human, token usage, department attribution for chargeback, failing prompts and how many function calls each iteration took. Trivedi summed it up: "observability in an agentic system is like a flight recorder." A log shows which API was called, he said, but the goal is to understand why the decision was made. Bhat pushed further, saying observation on its own amounts to monitoring and that the industry is moving toward fixing problems automatically.
Where should humans sit in the loop?
Trivedi said oversight should match an action's risk and the agent's confidence. "The goal isn't to keep a human in the loop. It is to make sure the human is a step away from the loop where it matters." His example was tuition reimbursement: an agent can check eligibility and approved courses, while a person approves the payment.
Altman was skeptical of treating humans as review machines. Posit's tools are used for analysis where correctness matters, such as clinical trial data, she said, and it is easy for people to approve agent-written code without reading it, especially when they lack the context the agent built up. Posit limits tool calls on some tasks so humans can keep up.
St Vil said some customers want humans focused on approving improvements the system proposes to its own workflows and policies, which also reduces approval fatigue.
How do you keep agents within their intended scope?
Trivedi called for a control tower that shows each agent's path through data and other agents, with a kill switch to stop a task in progress. Bhat applied his teammate analogy: "We give them keys to specific rooms, never the master key." Rules, he said, should be enforced at the tool boundary, not inside the reasoning layer.
What breaks between prototype and production?
Trivedi said teams need rich, well-integrated data in the sandbox, using anonymized production data, so mistakes surface before production. Altman pointed to business logic that lives in data teams' heads, such as which of two similar columns to use, and to the jump from a prototype where everyone has the same access to production users with different permissions. Bhat named unclear ownership, unrepresentative demo data and cost, which he said teams routinely overlook.
To keep systems aligned, Altman recommended running evaluations whenever a new model is swapped in, since models keep changing.
Join the next conversation
Data Science Connect's 2027 schedule includes 24 free live webinars in four series, among them the AI × Enterprise Panel Series of expert discussions and the Agentic AI Showcases. Registration is coming soon; see the webinar schedule for upcoming sessions.
Frequently asked questions
How do you govern autonomous AI agents?
Trivedi recommended tight scoping, least-privilege permissions for each agent, a policy layer that sits outside the agent's reasoning, and tiered approvals that require a human for high-risk actions.
When should a human approve an AI agent's action?
Panelists said it depends on risk: irreversible actions, payments, personal data and external communication need approval, while routine low-risk steps can run on their own.
What does observability look like for multi-agent systems?
Panelists described giving agents identities with scoped tokens, then tracing prompts, users, token usage and tool calls end to end so teams can see why a decision was made.
What blocks AI agents from moving from prototype to production?
Unrepresentative sandbox data, business context that lives only in people's heads, user-level permissions, unclear ownership and unplanned costs, according to the panel.
Watch: Operationalizing Agentic AI: Governance & Observability with Rubrik, Posit, Revefi, SAP & AT&T on YouTube
Event: Webinars · More coverage: Webinars insights
