Cheaper Tokens, Bigger Bills: Why AI Agent Costs Are the Next Budget Fight

For two years, enterprise AI teams have been told that the price of intelligence is falling. Token prices keep dropping, and each model generation does more for less. This week Gartner offered a less comfortable view: the cost of getting work done with AI agents is heading up, not down. The research firm predicts that inference costs per agentic workflow will rise more than fivefold through 2028, even as the price of individual tokens falls, Techstrong.ai reported. Within days, vendors from Snowflake to TrueFoundry were selling tools to keep that bill in check. For CIOs and CFOs, the week marked a turn: agent costs are becoming a budget line that needs its own management.

Why are AI agent costs rising when token prices fall?

Gartner calls it an inference paradox. Cheaper tokens encourage companies to adopt more capable models and more ambitious uses, and those consume far more tokens and compute, pushing total spending up, according to Techstrong.ai. Sending a task to an agentic reasoning model can cost a provider at least five times as much as a simple chatbot exchange.

The difference lies in how agents work. A chatbot answers a prompt once. An agent reasons through a task, calls tools, checks the results and decides what to do next, often looping many times before it finishes. Each step consumes tokens.

That is why Will Sommer, a senior director analyst at Gartner, argues product leaders cannot count on cheaper tokens alone to make advanced AI affordable, Techstrong.ai reported. In his view, general-purpose autonomous intelligence can cost far more than a carefully routed combination of models chosen for each job.

The forecast sits alongside earlier Gartner predictions that point in different directions. In March, the firm said running inference on a trillion-parameter model will cost providers more than 90% less in 2030 than in 2025. In June, it predicted AI coding costs will exceed the average developer's salary by 2028, as token use rises and vendors shift from per-seat to consumption pricing, according to the same report. The unit gets cheaper while the volume grows faster.

What is model routing, and can it cut AI costs?

The most direct response so far is to stop sending every task to the most expensive model. Gartner's first recommendation is what it calls inference tiering: route each step of a workflow to the model matched to what that step needs, saving frontier models for the hard parts.

Snowflake moved in that direction on Tuesday. The company announced dynamic model routing for its Cortex AI Gateway, which picks a model for each task by balancing quality against cost. Simpler or repetitive work goes to more efficient models, and deeper reasoning goes to frontier models. Customers can also restrict which models and providers their users can reach, which matters for regulated industries.

Snowflake's own figures show what is at stake. In an internal test, agents building a dbt data pipeline achieved up to three times better token efficiency than a frontier-model-only approach at the same quality, the company said. In a separate test, engineering teams completed the same number of pull requests with 25% better token efficiency. Those are vendor tests, and the routing feature is headed to private preview rather than general availability. "Snowflake's role is to absorb that complexity so customers can focus on outcomes," CEO Sridhar Ramaswamy said in the release.

AI Connect speakers have pushed the same right-sizing idea for some time. Mistral AI offers families of models instead of one large one because the best return comes from "the right model at the right time in the right place," Aaron Delp, the company's global head of AI technical marketing, said during a AI Connect webinar on building enterprise agents with right-sized models.

How are vendors helping enterprises control agent spending?

Routing is one of several controls that arrived this week. TrueFoundry launched TrueForge, an open-source agent harness that applies guardrails, access policies and budget controls across any model or MCP server, according to a Solutions Review roundup of the week's AI news. NeuBird AI published an open framework under which agents earn write access only after proving reliable performance, the same roundup noted.

Pricing models are shifting too. Globant introduced Glob.AI, a self-service way to buy AI services priced by output or consumption, while Sauce Labs opened its AURA product to customers who want to bring their own open-source, open-weight or proprietary models, RTInsights reported. Both moves give buyers more say over which models do the work and how they pay for it.

Gartner's other recommendations are less about products than about discipline, according to Techstrong.ai: set token limits, especially for coding agents that overuse tokens when given too much autonomy or context; monitor usage to flag high-consumption workflows; require reasoning agents to earn substantially higher returns than basic models; and press vendors to explain how they calculate and bill tokens.

What should CIOs and CFOs do about rising inference costs?

None of this is new to practitioners who have lived through cloud bills. At COLLIDE 2024, Varun Tottempudi, head of data and platforms engineering for global analytics at Lockton, warned that without monitoring, auditing and alerts, "in six months you might see a bill" as large as the past five years' spending, in a session on AI ROI across the project life cycle. Agents raise those stakes, because they decide on their own how many steps a task takes.

The practical shift is to measure cost per completed outcome rather than cost per token. Techstrong.ai's reading of the Gartner research makes the same point: the price of finishing a workflow matters more than the price of a single token. Amey Desai, head of AI at Nexla, put it simply when closing a AI Connect panel on scalable, cost-effective RAG, urging teams to "calculate the cost per question."

For enterprise leaders, that suggests a short checklist. Know what each agent workflow costs to complete, and what value it creates. Route routine steps to smaller models by default. Put spending caps and alerts in place before scaling agents, not after the first surprise invoice. And treat vendor pricing terms as part of architecture review, since consumption-based pricing makes costs harder to forecast.

Gartner's fivefold figure is a forecast, not a certainty, and it rests on how fast enterprises push agents into complex work. But the direction is clear enough. As agents move from demos to production, the companies that manage cost per outcome, not just cost per token, will be the ones able to keep scaling them.

Frequently asked questions

Why are AI agent costs rising if token prices are falling?

Agents reason, call tools and loop through many steps, using far more tokens per task, and cheaper tokens encourage more ambitious uses. Gartner calls this an inference paradox.

What is AI model routing?

Automatically sending each task or workflow step to the cheapest model that can handle it well, reserving frontier models for complex reasoning.

How can enterprises control AI agent spending?

Gartner recommends inference tiering, token limits, usage monitoring, requiring higher returns from reasoning agents and transparency on how vendors bill tokens.

Filed under: AI Strategy & ROI, Agentic AI, Data & AI Infrastructure