Harvey's Margin Swing Shows Why AI Startups Are Turning to Open Models

Harvey, the legal AI company valued at more than $15 billion after a September funding round, saw its gross margin fall from about 50% at the start of 2026 to roughly negative 50% by June as customers used its agents far more than expected, Bloomberg reported this week, citing a person familiar with the matter. The fix was not a price increase. It was a model of its own, post-trained on an open-weight base from China's Moonshot AI. Harvey is not alone, and the shift matters to every enterprise that buys AI through a software vendor, because the model inside the product is quietly becoming a cost decision rather than a brand decision.

What happened to Harvey's margins?

According to Bloomberg's report, as summarized by The Next Web, Harvey's token usage rose about twentyfold this year, by the company's own account, with a jump after a March update to its agents. Harvey had built its products on customized models from Anthropic, OpenAI and Google. As its customers ran more agentic work, the cost of serving them outran what they paid.

On August 27, Harvey released Harvey Tenet, its first in-house model, post-trained on Moonshot's open-weight Kimi K3. At the time, Harvey said the model beat leading US frontier systems on long-horizon legal agent tasks and pointed to the lower per-token prices of open-weight models, the South China Morning Post reported. Its gross margins turned positive afterward, people familiar with the work told Bloomberg. Harvey declined to comment on specific financials. The Next Web also reported that Anthropic had shown investors a slide about startups building their own models, while noting that Harvey still relies on Opus for its hardest tasks.

The story is less about one company than about a business model under strain. Many AI application companies sell seats or subscriptions, then pay model providers by the token. Agents break that math, because a single request can trigger dozens of model calls.

Which other companies are moving to open-weight models?

Harvey has company, according to The Next Web. It pointed to clinical documentation company Abridge, which is building a model on Nvidia's open weights, and customer service company Decagon, which it said routes 80% of customer queries through its own models. Finance software company Ramp is weighing training its own models, and co-CEO Karim Atiyeh said the idea would have made no sense a year earlier.

Earlier in September, Cognition took the same route. Cognition said on September 10 that its SWE-2 coding model is post-trained from Kimi K3 and comes within one point of Anthropic's Claude Fable 5.1 on the FrontierCode benchmark while costing 64% less, though it trails Fable 5.1 by a wide margin on Terminal-Bench 4.0. Those are the company's own figures.

The supply of capable open models keeps growing. On September 22, Xiaomi released MiMo-V2.6-Pro and MiMo-V2.6-Flash under the permissive MIT license, with weights, a technical report and reinforcement learning code on Hugging Face, Let's Data Science reported. The license allows self-hosting and modification.

Why are the frontier labs cutting prices at the same time?

The closed-model providers are responding. On September 22, Anthropic launched Claude Opus 5.5 at 20% lower list prices than Opus 5 and estimated a 40% saving on typical workloads, the company said. OpenAI released GPT-6 Sol and Luna the same day at about half the price of the models they replace, Techzine reported. Constellation Research's Larry Dignan listed open-weight models among the forces pushing Anthropic to compete on price, and expects open models to stay in enterprise use, either customized in-house or delivered through SaaS vendors.

There is a counterpoint in the same week's news. Open weights do not guarantee cheap service forever. DeepSeek, whose models are openly released, raised its API prices by 2.3 to 4.5 times depending on the model after warning developers in August, and its annualized revenue run rate has since passed $1 billion, The Information reported, according to The Next Web. Founder Liang Wenfeng told investors customers stayed after the increase. Hosted pricing for any model, open or closed, can move.

Is building on your own model the right call?

Not everyone is convinced. Menlo Ventures partner Matt Kraning called the question of owning a model "the wrong question" and suggested many such efforts are more show than substance, The Next Web reported. Post-training and serving a large model take talent, GPUs and ongoing evaluation work that most companies do not have.

AI Connect speakers have made both arguments. "This isn't renting AI. This is owning your AI," Mistral AI's Aaron Delp said during a AI Connect webinar on why enterprises should own their AI, stressing that the customer should control the data, the weights and the keys. In an earlier AI Connect panel on self-hosted LLMs and vendor lock-in, Alteryx Principal Engineer Jonathan Lounsbury offered the caution. "Hosting your own models is a really big investment," he said, advising teams to prove use cases with major providers before moving workloads in-house.

The likely outcome for most enterprises sits between the two. Few will train models, but many will run a mix: frontier models for the hardest tasks, cheaper open or mid-tier models for the bulk of agent traffic, and routing in between. Verisk Global Chief Data Officer Louis Dimodugno warned at The AI Enterprise Conference 2026 that token prices change almost daily and could rise once frontier providers go public, part of his case for a self-funding AI strategy.

What should enterprise buyers ask their AI vendors?

Harvey's experience gives buyers three practical questions. First, which models power the product, and can the vendor change them without notice? Many enterprise contracts never say. Second, how is the vendor's own model cost exposed in pricing, and what happens to usage-based fees if the vendor's margin turns negative? Third, if a vendor moves to a post-trained open model, how was it evaluated on work like yours, and where is it hosted?

For internal teams, the lesson is to treat models as replaceable parts. Keep evaluation sets for key workflows, log cost per completed task and keep prompts and tools portable. Harvey's recovery came from knowing exactly what each task cost it. Enterprise AI programs will need the same discipline as agent usage grows.

Frequently asked questions

Why did Harvey build its own model?

Bloomberg reported that rising token usage pushed Harvey's gross margin negative; after it launched Harvey Tenet, post-trained on Moonshot's open-weight Kimi K3, in August, margins turned positive.

What is an open-weight model?

A model whose trained weights are published so others can host, fine-tune or post-train it, subject to its license, such as Xiaomi's MIT-licensed MiMo-V2.6.

Should enterprises train their own models?

Most should not. A mix of frontier models for hard tasks and cheaper models for routine work, with portable prompts and evaluation sets, is more practical.

Filed under: AI Strategy & ROI, Data & AI Infrastructure, Agentic AI