Most organizations struggling for AI returns lack readiness, not technology. At COLLIDE 2025, Elder Research's Jennifer Schaff used a football analogy and client stories to explain eight gaps that quietly erode ROI, from strategy misalignment and fragmented analytics to failed adoption and weak data literacy.
Key takeaways
- Draw a straight line. Insights should trace directly to top business objectives, or they become trivia.
- One definition per metric. Fragmented analytics teams produce conflicting numbers and distrust, as when silos defined a blood donor differently.
- Prioritize ruthlessly. Spreading effort across many projects blurs ownership and slows results.
- Involve the front line. A manufacturing optimization failed because plant workers weren't consulted and didn't trust the data.
- Buy tools for defined problems. Schaff described a tool that cost a million dollars a year and went unused for five years.
Organizations struggling to get a return on AI usually have a readiness problem, not a technology problem, Elder Research Vice President of Commercial Services Jennifer Schaff told attendees at COLLIDE Data + AI Conference 2025 in Atlanta, in a session sponsored by Elder Research. Schaff, whose firm provides AI and analytics enablement consulting, laid out eight common gaps that quietly erode ROI and a diagnostic question leaders can ask about each.
Why are companies still struggling to get ROI from AI?
Schaff opened with a familiar scenario: an AI inventory system meant to optimize performance is more than 10% off, and no one knows whether the data, the model or the problem definition is to blame. She said surveys have long put analytics project failure at 80% to 85%, and she cited MIT research projecting that up to 95% of organizations report zero return on AI.
She welcomed the industry's shift from talking about failure to talking about ROI. "Failure" is a loose term, she said. A proof of concept that shows an idea would cost more than it returns, and is dropped quickly, is arguably not a failure at all.
Plenty of good frameworks already exist, Schaff said, so she set out to explain why some organizations win while others struggle. She used a football analogy: winning teams differ in jerseys and playbooks but share strong leadership, execution, experience, healthy players and trust. Struggling teams show five weaknesses that map to eight analytics gaps.
How does strategy misalignment hurt AI ROI?
Lack of leadership shows up first as strategy misalignment. Schaff's diagnostic question: "Can I draw a straight line from our top business objectives to the insights that we produce each week?"
She gave an example from early in her career. A client handed her team paid search data, and they identified changes worth about $6 million a year in savings plus more website traffic. The director couldn't use it, because a third-party agency managed paid search and marketing optimization wasn't a corporate objective that year. The result of misalignment, she said, "is really generation of trivia."
The second leadership gap is fragmented analytics, when departments build their own teams without a shared vision. At Canadian Blood Services, Schaff said, analysts from different silos discovered they defined a blood donor differently: someone who gave in the last six months, in the last year, or more than once. Multiple sources of truth breed distrust and slow enterprise adoption.
What execution and data problems erode AI returns?
Sloppy execution covers two gaps. The first is scattered effort from poor prioritization. Schaff said she has never met an organization that could pursue every proposed idea. Pouring effort into one pitcher makes ownership clear, she said, while spreading it across many small cups leaves things spilling and no one sure who owns what.
The second is using AI without understanding it. Schaff cited a 2016 book on the dangers of opaque algorithms and its account of a well-loved teacher fired after a statewide algorithm rated her ineffective without accounting for her students' circumstances. Misused AI can cost money, jobs and worse, she said.
Poor or incomplete data is the injury on the team. Schaff described asking ChatGPT for a picture of a great basketball player and getting one with four arms, a reminder that flawed data produces results no one should act on.
Why do AI solutions fail to get adopted?
Failure to adopt reflects a team without both offense and defense. Schaff described a manufacturing project that resequenced products on a line to increase machine uptime. The idea came from the corporate office, but plant workers were never consulted. They didn't trust the manually entered data, and some new sequences weren't feasible because of raw material delivery timing and how machine components were stored.
How do data literacy and tool choices affect AI ROI?
Lack of experience shows up in two final gaps. One is right-sizing technology. Schaff described a client that bought a software-as-a-service tool after three executives each proposed a different use for it. Five years later, she said, it had cost a million dollars a year and, as far as she knew, still wasn't being used. Without a defined problem, "it just becomes tech clutter and wasted dollars."
The other is data literacy among decision makers. One company saw a 15% sales lift from a regional marketing test, rolled it out nationwide, and watched it fizzle. The lift had come from a competitor closing its stores in the test region. Data literacy means interrogating the story behind the numbers, Schaff said. "Otherwise, you're just chasing illusions and not ROI."
Her prescription has three steps: spot the gaps, address them systematically to clinch execution, and bring everyone along. "Analytics and AI as you all know doesn't work in a back room," she said.
Discuss AI strategy and ROI at COLLIDE 2027
COLLIDE, the largest AI conference in the Southern U.S., brings more than 1,000 enterprise data and AI leaders to Atlanta for sessions on turning AI investment into results. The 2027 conference takes place Wednesday, October 13, 2027, at the Sandy Springs Performing Arts Center. More details are at COLLIDE, and Data Science Connect's AI × Enterprise Panel Series is part of its free live webinars.
Frequently asked questions
Why do so many AI projects fail to deliver ROI?
Schaff said the cause is usually a lack of foundational readiness, such as misaligned strategy, poor prioritization, bad data, low adoption and weak data literacy, rather than the technology.
How can leaders check whether AI work is aligned with strategy?
Schaff suggested asking whether you can draw a straight line from top business objectives to the insights the team produces each week.
What is fragmented analytics?
It happens when departments build their own analytics teams without a shared strategy, leading to different definitions, mismatched numbers and distrust in the data.
Why does data literacy matter for AI ROI?
Decision makers who can't interrogate results may act on misleading findings, as in a regional campaign whose sales lift actually came from a competitor closing stores.
Watch: If AI Is So Smart, Why Are We Still Struggling for ROI? on YouTube
Event: COLLIDE AI Conference · More coverage: COLLIDE AI Conference insights
