Garbage In, Liability Out: Data Quality and Governance for Trusted AI

Watch the full session · 39 min

AI is only as trustworthy as the data and governance behind it, a COLLIDE Data + AI Conference 2026 panel agreed. Leaders from Bank of America, Synchrony, Alteryx and Truist discussed fixing data at the source, measuring whether AI is fit for purpose, rethinking model risk governance and naming a single accountable owner.

If a company can't trust its data, it can't trust its AI. That was the starting point for "Garbage In, Liability Out: Data Quality, Governance, and AI You Can Trust," a panel at COLLIDE Data + AI Conference 2026 in Atlanta that brought together data, risk and analytics leaders from Synchrony, Bank of America, Alteryx and Truist.

Anchi Zo, who leads the fair banking compliance team at Truist, moderated the discussion with Jacob Kosoff, a model development executive at Bank of America; Charity Borrows, who leads data governance and data quality oversight at Synchrony; and Jason Klein, Director of Portfolio Marketing at Alteryx. Zo noted that panelists' comments did not reflect the views of their organizations.

What does trustworthy AI look like in practice?

"I would define trustworthy AI is AI that is fit for purpose," Kosoff said. His example was customer complaints. Most large banks, he said, now use AI to summarize contact center calls and identify complaints, work employees used to do by hand.

To measure whether that AI is trustworthy, Kosoff said banks track how often employees edit, change or discard the AI's output, and how often quality review teams agree with the final result. That shows where the tool works and where humans routinely override it. He said banks have also started sitting with employees to watch how they use tools such as AI that drafts credit approval memos or suspicious activity reports.

Borrows said trust has to hold across the full chain, from data source to model to output to business outcome. She described three parts: trusted data, trusted governance and trusted decisions. "If you can't trust your data, you can't trust the AI," she said.

Klein offered Alteryx's test, which he called VURA: is an AI answer visible, understandable, repeatable and auditable? Repeatability is where many teams struggle, he said. If the same question gets a different answer an hour later, people stop trusting the system.

Why fix data quality at the source?

Borrows said data issues should be fixed at the source, with ownership and accountability assigned there, rather than corrected at each hop or at the target. Her team sets data quality rules at the source and keeps controls downstream as well.

AI won't fix missing data or incomplete lineage, she said. Those problems need solutions before an AI layer is introduced.

Can you point an LLM directly at enterprise data?

Analytics is moving from dashboards into AI assistants, Klein said, and people ask whether they can simply put an LLM on their data. Alteryx highly recommends against that, he said, citing cost, answers that vary each time and speed. He recommended running deterministic, governed analytics first and passing only certified results to AI. Klein said benchmark tests found the Alteryx approach used up to 93% fewer tokens and ran 85% faster.

Borrows added that data can be accurate and still not be fit for a given AI use case, which is why human review matters.

Kosoff described a peer institution that launched an internal "ask data" chatbot. A credit risk modeler asking about customer income found several kinds: stated income on credit card applications, estimated income and verified income for mortgages. The data wasn't wrong, Kosoff said, but employees might not know which figure they were getting, or that the differences existed.

How are banks rethinking AI governance?

Kosoff said many financial firms are reimagining governance after new model risk guidance, which he identified as SR 26-2, replaced SR 11-7, the 2011 model risk management guidance he described as very prescriptive. The new guidance gives institutions more flexibility, he said.

Firms are drawing clear swim lanes among model risk, compliance, legal, operational risk and technology risk, often with a RACI approach, so three risk officers don't ask a business owner the same question. Validation is also shifting from fixed schedules, such as every two years for high-risk models, toward triggers based on performance. Some fraud teams now get daily feedback on false positives and false negatives, he said.

He also urged teams to drop monitoring metrics that exist only to satisfy auditors. His test: "If something goes red, would you change anything?" If not, it is the wrong metric.

Klein said the industry's language has shifted from "self-service" to terms like "governed self-service" and "bounded autonomy." In Alteryx's model, IT owns the platform and data, business teams own the business logic, and access permissions set by IT carry through to the AI system.

Who is accountable when AI goes wrong?

The line of business that uses an AI tool owns it, Kosoff said, not the tech, modeling or data quality teams. For a tool that drafts credit approval memos, that is the credit team, with one named owner.

Ownership lets that owner set their own guardrails, he said, such as limiting a tool to underwriters with at least two years of experience who would recognize a flawed draft. If a named owner isn't taking the role seriously, he said, the organization should replace them.

The panel's one-line advice

Asked for one principle for turning AI from a liability into a trusted system, Kosoff said, "Ownership." Borrows said, "Trust the data, govern the decision." Klein said, "VURA."

More on AI governance at COLLIDE 2027

COLLIDE, the largest AI conference in the Southern U.S., brings more than 1,000 enterprise data and AI leaders to Atlanta. The 2027 conference takes place Wednesday, October 13, 2027, at the Sandy Springs Performing Arts Center. Between conferences, Data Science Connect's free AI × Enterprise Panel Series webinars continue the conversation on governance and data quality.

Watch: Garbage In, Liability Out: Data Quality, Governance, and AI You Can Trust on YouTube
Event: COLLIDE AI Conference · More coverage: COLLIDE AI Conference insights