In the closing panel of COLLIDE 2025, data and AI leaders from The Home Depot, Equifax and Mars Wrigley explained how they measure AI return on investment. They tie use cases to business OKRs, trade off speed, accuracy and cost, and argue AI belongs in the middle of the process, not first.
Key takeaways
- Start with the OKR. Measure AI by its impact on the business objective it supports, using a metric as close to that outcome as possible.
- Reuse the foundations. Teams can't scale if every use case starts again from data access and identity stitching.
- Trade-offs depend on the use case. Customer-facing AI needs higher accuracy than internal insights, and early pilots should favor speed within guardrails.
- Ground the executives. Data leaders now spend more time explaining what AI can realistically do than showing what it might do.
- AI in the middle, not first. Fix the business process and try simpler analytics or automation before reaching for models.
Measuring AI return on investment starts with the business goal, not the model, according to data and AI leaders from The Home Depot, Mars Wrigley and Equifax. In the closing panel of COLLIDE Data + AI Conference 2025 in Atlanta, Mustafa Nisar, Senior Director, CX Data Science and Analytics at The Home Depot, and Sudepto Ghosh, who leads internal data analytics and AI at Equifax, discussed how they tie AI to results. Shashank Kadetotad, who leads the enterprise data science and AI team at Mars Wrigley, moderated.
The panelists returned to one idea throughout. "AI is a means to an end. It's not an end in itself," Nisar said.
What should you measure when evaluating AI ROI?
For Nisar, the most important measure is the impact on the business OKR the AI initiative is meant to support. Teams should find a metric as close to that outcome as possible, he said, so they measure AI as a facilitator of the result rather than measuring AI for its own sake.
If the hypothesis isn't measurable, that is where the work has to start, Nisar said. A goal like improving the e-commerce returns experience means little until a team defines which part of the experience it will change and how.
Ghosh gave a concrete example from dispute resolution. Case volumes per week or month are measurable, so his team looks at whether AI reduces time to resolution. It is also giving employees chatbots so they can resolve their own cases, with a person brought in for whatever is left.
How do enterprises scale AI beyond one use case?
Scale depends on reusable foundations, Nisar said. "I want to avoid working on step zero and step one every time we're executing or working towards a use case," he said. Step zero is access to the right data signals. Step one is stitching them to identifiers such as a customer and modeling them.
Ghosh described two kinds of scale at Equifax: tools that help functions such as finance close the books faster at month end, and horizontal tools such as summarization that reach users in sales, HR and other groups.
How do you balance speed, accuracy and cost in AI?
Kadetotad said enterprise users now expect consumer-grade response times. If a public chatbot answers in a few seconds, an internal chatbot can't take much longer than five seconds for a complex question. His team leans into speed first and then optimizes for cost as it scales.
Ghosh said LLM costs need close monitoring because teams often don't know how many tokens a given prompt or query consumed. On accuracy, he described a working target of around 90% to 95%, measured through thumbs-up and thumbs-down feedback.
Nisar said the right trade-off depends on the use case and its stage. An internal marketing insight reviewed by an analyst needs a different accuracy threshold than an autonomous chat with a customer. Early in a proof of concept, speed matters most, as long as guardrails limit the risk.
How do you set expectations with executives about AI?
Leaders hear from many voices, including large cloud vendors, Ghosh said, so data teams need to understand that context, explain what is approved for use internally and document quarter-by-quarter delivery commitments.
Kadetotad said the job has changed. Data teams used to show what AI could do aspirationally, but now "our job has become grounding the leaders on what actually can be done." Mars uses its North America AI lab, with physical and virtual space and cloud credits, to let technical staff experiment and fail fast.
Nisar argued that "an AI or data science leader should be a business leader first," someone who understands each department's strategy. Ideas also need to be tangible, he said, because abstract proposals are hard to tell apart from dozens of others.
What role do pilots and proofs of concept play?
Nisar called pilots critical. The harder question is funding them while teams are committed to existing roadmaps. He recommended agreeing with business partners that AI teams spend a partial allocation exploring ideas tied to those roadmaps.
Ghosh said his team constantly grooms its technical roadmap because cloud AI services change quickly. He described a period when many proofs of concept never saw the light of day, and said company-wide AI and analytics challenges are now generating ideas.
What would these leaders do differently?
Ghosh would invest earlier in the data foundation and in source-system metadata, which he said drives prompting and context for generative AI. Data should arrive "nice, fresh and healthy like breakfast," he said. He also pushed back on AI-first thinking. "AI should be in the middle, not first," he said, urging teams to fix business processes and try simpler analytics or automation before reaching for models.
Nisar would build a modular architecture so components can be swapped as leading language models change. "We don't need to be married to one," he said. Kadetotad pointed to breaking down silos as the element he is most purposefully pursuing.
See sessions like this at COLLIDE 2027
COLLIDE, the largest AI conference in the Southern U.S., brings more than 1,000 enterprise data and AI leaders to Atlanta for a day of keynotes and panels on putting AI to work. The 2027 conference takes place Wednesday, October 13, 2027, at the Sandy Springs Performing Arts Center. Registration is coming soon. More details are at COLLIDE.
Frequently asked questions
How do you measure the ROI of an AI project?
Panelists recommended tying each use case to the business OKR it supports and choosing a measurable metric close to that outcome, such as time to resolve a dispute.
How accurate does enterprise AI need to be?
It depends on the use case. Equifax's Sudepto Ghosh described a working target of around 90% to 95% for internal chatbots, while Home Depot's Mustafa Nisar said customer-facing AI needs a higher bar.
How should companies fund AI proofs of concept?
Nisar suggested agreeing with business partners that AI teams spend a partial allocation exploring ideas tied to existing roadmaps, so pilots don't become extra unplanned work.
Should companies take an AI-first approach?
Ghosh argued AI should be in the middle, not first: get business processes right and consider automation or analytics before building with models.
Watch: AI ROI in Practice: What Leading Enterprises Get Right on YouTube
Event: COLLIDE AI Conference · More coverage: COLLIDE AI Conference insights
