Mistral AI's Aaron Delp argued that enterprise AI fails when it ignores domain knowledge and chases benchmarks. He demoed no-code agents, OCR-powered parts search over MCP and an agentic drug submission workflow, and explained why right-sized and custom models matter.
Key takeaways
- Benchmarks aren't business value. Delp warned that leaderboard claims fade in days while ROI depends on domain intelligence.
- Unstructured data is the main hurdle. OCR and speech models turn documents and recordings into data AI can use.
- Right-size the model. Small, specialist and large models each fit different jobs; 'you don't need the Ferrari' for routine tasks.
- Agentic workflows can pre-check regulatory filings. An eCTD workflow flagged gaps module by module before submission.
- Customization is becoming the edge. Delp said custom models are becoming one of the only competitive advantages, and Forge aims to lower the risk.
Enterprise AI has had "a little bit of a mixed reception," Mistral AI's Aaron Delp said, because most off-the-shelf AI ignores what makes each organization different: its data, processes and people. In the Data Science Connect webinar "Building With Mistral: Open, Powerful and Ready to Scale," sponsored by Mistral AI, Delp, Global Head of AI Technical Marketing, made the case for right-sized, customized models and showed demos at three levels of technical depth.
Why does off-the-shelf AI underdeliver for enterprises?
Delp pointed to the MIT study estimating that 95% of enterprise AI use cases fail to reach production. Part of the problem, he said, is "benchmaxing." Models are tuned to top leaderboards, he said, and are even starting to recognize when they are being benchmarked, yet their state-of-the-art claims last days or sometimes hours.
"AI should move the needle for your company," he said, by cutting costs and moving the business forward. That requires deep domain intelligence, controllable deployments and a feedback loop like those familiar from DevOps: ingest domain data, train or fine-tune a model, deploy it, capture results and keep tuning.
He named unstructured data as one of the biggest obstacles. It sits in Google Drive, OneDrive, Slack, file shares, CRM and ERP systems, and has to be brought together before it becomes useful to AI.
How can teams start building AI agents without code?
Delp's first level needed no code. In Mistral's chat assistant, he launched the agent builder, picked a built-in data analyst agent, set its personality to polite, mature and concise, attached connectors to Google and Microsoft services, and had it analyze a public CSV of New Zealand industry data. Spreadsheets matter, he said: "CSVs still run a lot of enterprises."
He then transcribed a video file with Mistral's audio model, which labeled speakers and translated the transcript into French. The interface accepts up to 10 files at a time, he said, while the API handles up to a million in batch mode and generates ready-to-use Python, TypeScript or curl code.
How does OCR plus MCP improve enterprise search?
At level two, Delp used AI Studio to run OCR on an engineering drawing, outputting Markdown, HTML or JSON based on field descriptions he supplied. The results went into a parts database.
He then asked, through an MCP connector, for parts with a particular diameter. No part matched, but the model found a hole with that diameter in a drawing and asked whether he wanted details. A standard database query would have returned zero results, he said. Instead, the AI did "a little bit of problem solving" and surfaced an answer that normally would not have appeared.
Why choose a smaller AI model?
Mistral offers model families instead of one large model because the best return comes from "the right model at the right time in the right place," Delp said. Its generalist models range from 3 billion, 8 billion and 14 billion parameters at the small end to almost 700 billion at the large end, alongside open-weight and licensed options, specialist models for OCR, speech and code, and experimental labs models.
He compared the largest model to a supercar that is expensive and hard to maintain and customize. "If all you're doing is going to the grocery store, you don't need the Ferrari," he said.
In Q&A, he described agent swarms in the same terms: an organizer agent on a properly sized generalist model calls coding or specialist models for subtasks, passing inputs and outputs between them.
How can AI agents speed up regulatory submissions?
Delp's level three demo was an agentic workflow in AI Studio for electronic common technical document (eCTD) submissions, the packets drug sponsors file with the FDA or EMA. The workflow analyzed each module's 10 to 15 documents and reported a summary, findings, gaps and recommended improvements. Teams can iterate against the workflow instead of the regulator, he said, to get the submission right the first time and save time to market.
What makes custom AI model training hard, and how is it changing?
"Custom models are really becoming one of the only competitive advantages," Delp said. But fine-tuning has been out of reach for many organizations. Models are brittle, runs can fail expensively, tools tend to lock teams into one cloud, and the skills are scarce. He compared it to early Kubernetes and OpenStack, when products eventually "put the ninjas in the box."
That is the role of Forge, Mistral's model customization product, which Delp said Mistral had used internally with customers before announcing it as a full product at NVIDIA GTC. It runs on the customer's infrastructure. Asked about air-gapped environments, he said Mistral packages AI Studio and its tools quarterly to run fully on-premises, with refresh frequency as the trade-off.
For organizations unsure where to start, Mistral runs an "iconic use case" workshop, followed by a feasibility proof of concept he said typically takes two to four weeks. The biggest mistake he sees when choosing a model, he said, is underestimating customization, which leads to scope creep instead of a clear goal and realistic ROI.
See more enterprise AI demos
Data Science Connect's 2027 schedule includes 24 free live webinars across the AI × Enterprise Panel Series, AI Demo Days, Agentic AI Showcases and Plug & Play Series. See the webinar schedule for upcoming sessions.
Frequently asked questions
Why do enterprise AI projects fail to reach production?
Delp said off-the-shelf AI often lacks domain-specific intelligence and deployment flexibility and is optimized for benchmarks rather than business outcomes.
How do you choose the right AI model size?
Match the model to the task: Delp recommends small or specialist models for routine or narrow jobs and reserving large models for work that needs them.
Can Mistral AI run in an air-gapped environment?
Delp said Mistral packages AI Studio and its tools quarterly so they can run fully on-premises in air-gapped environments, with refresh frequency as the trade-off.
What is the biggest mistake when choosing an AI model?
Underestimating customization, Delp said, which leads to scope creep instead of a clear end goal and realistic ROI.
Watch: Building Enterprise Agents with Mistral AI | Open, Powerful, and Ready to Scale (Webinar Replay) on YouTube
Event: Webinars · More coverage: Webinars insights
