AI ROI Across the Project Life Cycle: Business Cases, Data and Cloud Costs

Watch the full session · 33 min

AI ROI is won or lost at every stage of a project. At COLLIDE, leaders from Bank of America, Carelon Health, Lockton and PRGX walked through business cases, the three E's of measuring value, data and training bottlenecks, and how unmonitored cloud and GPU spending erodes returns.

Return on investment in AI depends as much on managing costs as on generating revenue. At COLLIDE in Atlanta, a panel of data leaders walked through the life cycle of an AI project, from the business case to production, and where ROI is won or lost at each step. The panel featured Jacob Kosoff, Managing Director and Data Science Executive at Bank of America; Suzanna Kim, Director of Data Science at Carelon Health; and Varun Tottempudi, Head of Data & Platforms Engineering, Global Analytics at Lockton. Manish Dingare, Chief Artificial Intelligence Officer at PRGX Global, moderated.

Dingare opened by recalling a former boss's warning: if IT projects were estimated accurately, half of them would never get funded. Add data, AI and machine learning, he said, and the complexity grows.

How do you build a business case for an AI project?

In financial services, Kosoff said, the strongest projects start with a line of business that has a real, costly problem, not a data science team with a new tool. His example was contact centers, where employees must help callers and also summarize calls and flag complaints. Doing both at once distracts from the customer, and doing it afterward makes other callers wait.

Banks tend to want the simplest approach that gets the job done, he said. "If you build me a spaceship, I'm not going to be very happy," he said, when the request was a vehicle to get to the supermarket.

The math also has to work. Spending thousands of data scientist hours to automate a task that takes 100 hours a week, plus GPUs and infrastructure, doesn't produce a positive return, Kosoff said.

What metrics measure AI ROI?

Kosoff measures internal AI tools, from GitHub Copilot to a call summarizer, with three E's:

1. Efficiency: Are employees saving time?
2. Effectiveness: Is the output better, with fewer errors, and do the model and employee perform better together?
3. Engagement: Do employees enjoy using the tool?

Engagement is new, he said. Two or three years ago, employee happiness rarely came up in investment discussions. Now leaders ask about it, and teams should challenge the assumption that a tool makes people happier by surveying them.

Another question he now hears in almost every meeting: is anyone else at the firm doing something similar? Leaders want teams to build on existing work, not duplicate it.

Kim said leaders also ask about time to market and whether six months to a year of development will pay off. Tottempudi said he is asked every three months how AI costs compare with day-to-day operational costs.

How does data infrastructure affect AI speed and cost?

Data is "the center of the universe" for AI applications, Tottempudi said. Teams need to know the sources, formats, scale and growth of their data and how it is curated for downstream use. When users of an application such as a claims tool see answers take one, two or three minutes, he said, the fix lies in how the data pipelines behind it were built. He called for automated data quality checks and profiling.

Kosoff shared a cautionary story from his time at Regions Bank, where his team validated a model that used income as an input. The bank held stated, estimated, verified and current income, each legitimate for its purpose. A developer assumed stated income from credit card applications was verified income, which caused problems. Data scientists need access to data dictionaries, metadata and the business, he said, and should feel empowered to ask simple questions.

What slows down AI model training?

Data movement is the biggest bottleneck, Kim said. Without parallel processing and distributed systems, feature engineering on large data sets can take a very long time. She pointed to Spark for distributed processing and Airflow for automated pipelines. Not every model needs a GPU, she added. A linear model doesn't, but deep learning does.

Dingare said he has seen teams train on entire data sets when a sample would do. Data scientists should understand FinOps and see what their cloud work costs, he said.

How do you balance AI infrastructure costs with ROI?

Without monitoring, auditing and alerts, Tottempudi warned, "in six months you might see a bill" as large as the past five years' spending. He said he has seen organizations rack up several hundred thousand dollars without knowing it. In his experience, a little over 78% of the AI models and virtual machines organizations run go unused, and only about 18% to 20% are operationalized.

Kim said cloud servers are easy to start and forget, so teams must stay focused on business goals. Kosoff added that sometimes basic lexicon enhancements can do the job instead of a large language model.

Dingare argued ROI works on both sides of the ledger. He said one team found its performance problem was in serving-time feature calculation, not the model, and saved 30% by fixing it. "In data center everything was free, and in cloud it is not," he said, so costs must be actively managed to move from proof of concept to production.

Get practical AI ROI lessons at COLLIDE 2027

COLLIDE, the largest AI conference in the Southern U.S., brings more than 1,000 enterprise data and AI leaders to Atlanta for panels like this one. The 2027 conference takes place Wednesday, October 13, 2027, at the Sandy Springs Performing Arts Center. Registration details are coming soon.

Watch: Best Practices for Infrastructure, Data Pipelines, and Operational Efficiency on YouTube
Event: COLLIDE AI Conference · More coverage: COLLIDE AI Conference insights