AI Hype to Real Results: Aflac, Carter's and DataRobot Panel
At COLLIDE 2023 in Atlanta, analytics leaders from Aflac, Carter's, DataRobot and RXA @ OneMagnify discussed why AI projects stall and how to choose ones that pay off. They described feasibility tests, governance and ethics, and what they expected from generative AI at the time.
By The AI Enterprise Magazine Team4 min read
Key takeaways
- Test feasibility, not just ROI. Carter's found top-down models became shelfware when sponsors could not act on the results.
- Start with low-hanging fruit. A medium-ROI, high-feasibility win builds credibility for harder use cases.
- Buy-in is a people problem. Showing how a model works and understanding the business earned trust.
- Keep humans in the loop. Usage shifts over time, so monitoring for drift matters for predictive and generative AI alike.
- Stay flexible on generative AI. Panelists advised against locking into a single ecosystem as tools changed daily.
At COLLIDE 2023 in Atlanta, a panel of analytics leaders from Aflac, Carter's, DataRobot and RXA @ OneMagnify argued that the hard part of enterprise AI is rarely the data science. In a session sponsored by DataRobot, a Platinum sponsor of the October 2023 conference, they described why early models stalled, how they pick projects now, and where they thought generative AI was heading. Colby Miller, a marketing manager at DataRobot, moderated.
The panelists were Selma Dogic, Senior Manager, Analytics COE at Carter's; Sarah Gibson, Analytics Manager at Aflac; Jason Harper, Managing Director at RXA @ OneMagnify; and Jessica Lin, Lead Data Scientist at DataRobot.
Why do enterprise AI projects stall?
Stakeholder buy-in came up first. Gibson said that at Aflac, "a very old company," winning over the business was one of the hardest challenges. Showing how a model works, and how its features influence volume projections, was the best way to get there.
Dogic added that the hype around generative AI creates pressure to "move fast," which can put "the cart before the horse." She said buy-in has to be real: stakeholders should care because of business impact, not because AI seems important.
Harper said clients span a wide spectrum, from leaders who want a task force out of fear of missing out to those who say they are not ready. His advice to the first group was to step back and ask what the organization is trying to achieve. To the second, he said, "you need to start making progress now."
How do you choose AI projects that pay off?
Dogic described Carter's early lesson. Its first models, such as e-commerce traffic predictions and a marketing demand model, came top down from executives. They were built quickly without checking feasibility, and "ended up being shelfware," she said.
Her test now has two parts: expected return and whether the sponsor can actually change decisions based on the model. She said she would rather take a medium-ROI, high-feasibility project, and use that success to expand into harder use cases.
Lin suggested a question for scoping: if the solution were perfect, what would implementation look like? The output "cannot die in a PowerPoint presentation somewhere," she said, and the answer often shows which stakeholders or IT teams need to be involved.
What use cases delivered value?
Gibson described one of Aflac's first DataRobot models, a personalization tool for agents that suggests products to prospective customers based on customers like them. She said it is in production and that Aflac can see the return. The team was also working on a model to identify customers likely to leave, using signals such as no claims filed or many calls to the call center.
Harper pointed to propensity models in automotive, a considered purchase where granular behavioral data supports tailored communications. He said that kind of work, which he once tried to do in Excel, had become low-hanging fruit.
How do teams build trust, governance and ethics into models?
Dogic said the development is quick but the iteration takes time: testing insights against subject matter experts and ground truth. She said monitoring for data drift through MLOps tooling helps keep models accurate.
Lin stressed keeping a human in the loop and watching how people use a model in practice, which shifts over time. That applies to chatbots, she said, as much as to predictive models.
On ethics, Dogic named privacy and the real-world consequences of decisions, such as who gets an interview. Gibson said Aflac has a team that reviews such issues, and an early suggestive-selling model was reined in when reviewers rejected many of the inputs. "Nobody likes to be told no," she said, but the review would help in the long run.
Harper said RXA has walked away from work, and warned that AI-driven lifecycle marketing can move "beyond influence and into coercion." He called responsibility shared among tools, consultants and clients.
Where was generative AI heading?
Dogic said Carter's saw the near-term value in augmenting people's work, and hoped later to use generative AI to change how customers shop. Gibson expected claims processing and call handling at insurers to be early areas.
Lin called generative AI "the absolute Wild Wild West" and urged flexibility. In her view, the most value would come where generative and predictive models are layered together, which needs extra pipeline governance. Her advice: "don't get locked into a single ecosystem."
Harper said clients increasingly ask for a private model trained on their own data instead of a public chatbot, and that RXA had recently deployed its first one.
Where to hear more from enterprise AI leaders
The conversation about moving from AI hype to results continues in 2027. COLLIDE 2027, the largest AI conference in the Southern U.S. with more than 1,000 enterprise data and AI leaders, takes place Wednesday, October 13, 2027, at the Sandy Springs Performing Arts Center in Atlanta. Details are at /events/collide/. Registration is coming soon. Free live sessions are also on the 2027 webinar schedule.
Frequently asked questions
Why do AI projects become shelfware?
Carter's Selma Dogic said models built top down without a feasibility check went into production but did not change decisions, so they went unused.
How do you choose the first AI use case?
Dogic weighs expected return and whether the sponsor controls the decisions needed to realize it, preferring medium ROI with high feasibility.
How do you build stakeholder trust in AI models?
Panelists said to show how the model works, test insights against subject matter experts, and demonstrate that data scientists understand the business.
What ethical issues did the panel raise?
They cited customer privacy, bias in data, high-stakes decisions such as hiring, and marketing that moves from influence toward coercion.
This article is based on remarks by Selma Dogic, Senior Manager, Analytics COE, Carter's; Sarah Gibson, Analytics Manager, Aflac; Jason Harper, Managing Director, RXA @ OneMagnify; Jessica Lin, Lead Data Scientist, DataRobot; moderator Colby Miller, Marketing Manager, DataRobot at COLLIDE Data + AI Conference 2023 on October 3, 2023. More coverage: COLLIDE AI Conference.
Filed under: AI Strategy & ROI, People, Culture & Leadership, AI Governance & Trust
