At COLLIDE 2025, KMS Technology CTO Guy Merritt showed how machine learning, generative AI and agentic AI each solve a different problem in a legacy quoting workflow. He outlined a prepare, diagnose and implement path for enterprises and shared KMS's own results from AI-assisted software development.
Key takeaways
- Prove value, not technology. Merritt said the technology works; the question is whether it delivers value for your organization.
- No single AI does it all. Machine learning predicts, generative AI writes, and agentic AI adapts dynamically to new data.
- Prepare, diagnose, implement. Start with a defined business outcome, diagnose the workflow before choosing a fix, and use one shared framework.
- Governance is the foundation. Agents making decisions on uncurated or poisoned data will produce bad outcomes.
- Coding gains depend on who uses the tools. KMS saw 2.5 to 3 times gains for experienced engineers on new code and drops for juniors on legacy code.
Most enterprise AI projects that fail do so because of people and process, not technology, KMS Technology Chief Technology Officer Guy Merritt told attendees at COLLIDE Data + AI Conference 2025 in Atlanta. In a session sponsored by KMS Technology, Merritt walked through how a legacy business application can absorb three generations of AI, a three-step path from preparation to implementation, and how his own engineering firm has applied that approach to software development.
Merritt said he built his first AI engine, an expert system, in 1992. Because KMS advises clients on AI, he said, it has to be "drinking our own champagne."
Why do so many enterprise AI projects fail?
Citing Gartner, Merritt said 75% of enterprises were expected to adopt some form of AI, and 70% of deployed projects were failing, meaning they never reached production. "The roadside is littered with" proofs of concept, he said.
He rarely uses the phrase proof of concept with clients. With trillions of dollars going into the technology, he said, "the technology works." The real question is whether it delivers value for a specific organization, so KMS talks about proof of value instead. Failures, he said, usually come from the people side, where change management is critical, or from failing to re-engineer the workflow.
How do machine learning, generative AI and agentic AI work together?
Merritt used a configure, price and quote (CPQ) application as his example. It pulls from part specs, ERP pricing and contracts, and is still largely manual. Many companies add an AI overlay on top of such systems, he said, and each generation of AI does a different job. "There is no AI panacea," he said.
- Generation one, machine learning: finds patterns in past data to predict outcomes, such as seasonal price curves.
- Generation two, generative AI with retrieval-augmented generation (RAG): the LLM is trained on public data and doesn't know the company's private data, so RAG supplies current company information. In CPQ, it writes the analysis and narration for a quote.
- Generation three, agentic AI: adjusts pricing dynamically as new data arrives. If a storm disrupted parts supply from Taiwan, Merritt said, the machine learning model would keep predicting the same price, while a well-engineered agent could adjust pricing or find another supplier.
Merritt also advised against most companies building their own foundation model. By his estimate, one takes about six months and $10 million, and is out of date as soon as it's finished.
What are the three steps to bring AI into an enterprise workflow?
Merritt described three steps: preparation, diagnosis and implementation.
Preparation starts with the business outcome, such as reducing support tickets by 40%. Many clients haven't defined one, he said; their stated problem is often that the board says they have to do it. Preparation also means mapping the current workflow, using process mining to find bottlenecks, and starting small in a "crawl, walk, run" approach before expanding.
Data governance belongs here too. Clients call it boring, Merritt said, but skipping it is like putting a better roof on a house whose foundation is made of sand. It matters more with agentic AI, he said, because autonomous decisions made on uncurated data produce bad outcomes. He also warned about data poisoning, in which someone alters a data set so a retrained model gives wrong results.
Diagnosis works like a doctor's visit, he said: understand what is wrong before choosing a treatment. That includes deciding which parts of the AI stack to build, outsource or partner on, and assessing maturity against a model such as those from NIST, Gartner or KMS. "Virtually all AI strategies today are wrong," he said, because the tools change so quickly, so organizations should reassess often.
Implementation needs one shared framework across divisions, he said. When every team follows its own path, the organization spends more and can't compare results.
Why do AI coding tools help some developers and slow others down?
KMS has 60 to 80 AI-enabled software projects underway at any time, Merritt said, and productivity results vary widely. According to KMS data, code-generation tools work best for engineers with about eight years of experience who understand the domain and are writing new code of moderate complexity, where KMS has seen 2.5 to 3 times improvement. For junior engineers working on complex legacy code, productivity drops.
Averaged out, KMS has seen gains of up to 32%, Merritt said, below the 45% the industry promises. KMS measures productivity in story points per sprint.
How is KMS using agents across the software development lifecycle?
Merritt described a vision in which each engineer is paired with a personal digital twin, an agent that learns how its human works and handles routine tasks such as code generation. Because the twins are aware of each other across a project, he said, they can flag when another engineer has already built a function and avoid redundant work.
KMS is building its own agent orchestration framework with role-based agents for business analysts, developers and testers, and compares multiple LLMs to find the best fit for each task. He said KMS will no longer select any new tool for its software workflow unless it has a Model Context Protocol (MCP) interface. KMS also uses AI on client work only with the client's permission.
See sessions like this at COLLIDE 2027
COLLIDE, the largest AI conference in the Southern U.S., brings more than 1,000 enterprise data and AI leaders to Atlanta for practical sessions on moving AI from pilot to production. The 2027 conference takes place Wednesday, October 13, 2027, at the Sandy Springs Performing Arts Center. See the COLLIDE event page for updates, with registration coming soon.
Frequently asked questions
Why do enterprise AI projects fail?
Merritt said failures rarely come from the technology. They usually stem from people and change management or from not re-engineering the workflow, and many projects lack a defined business outcome.
What is the difference between machine learning, generative AI and agentic AI?
In Merritt's CPQ example, machine learning predicts prices from past patterns, generative AI with RAG writes quote narratives, and agentic AI adjusts pricing or suppliers as new data arrives.
What is a proof of value in AI?
A pilot designed to show that AI delivers business value for a specific organization, rather than a proof of concept that only shows the technology works.
Do AI coding tools improve developer productivity?
At KMS, results varied widely. Experienced engineers on new code saw large gains, junior engineers on complex legacy code got slower, and the average gain was up to 32%.
Watch: The AI Maturity Journey: Guiding Enterprises from Exploration to Excellence on YouTube
Event: COLLIDE AI Conference · More coverage: COLLIDE AI Conference insights
