Generative AI Beyond the Hype: Bill Franks on Hallucinations, Limits
At COLLIDE 2023, Bill Franks of Kennesaw State University said generative AI is effectively making up every answer, which is why it is right on well-documented topics and unreliable on obscure ones. He expected companies to differentiate by building application layers on commoditized models.
By The AI Enterprise Magazine Team4 min read
Key takeaways
- Different, not just bigger. Generative AI creates new examples of data rather than appending predictions to existing rows, Franks said.
- Hallucination is the default. The model produces probable answers, so well-documented topics tend to be right and obscure ones are often made up.
- Answers vary. The same question can return different responses, so replicating results later is not guaranteed.
- Use the right tool. Lookups and math need real data sources and engines; summaries and drafts suit generative models.
- Build on top. He expected models to be commodities and value to come from company-specific application layers and ensembles.
At COLLIDE 2023 in Atlanta, Bill Franks, Director of the Center for Data Science & Analytics at Kennesaw State University (KSU), gave a talk on separating generative AI's real impact from its hype. Franks explained why the technology took off, what it reliably gets right and wrong, and why he expects companies to build application layers on top of commoditized models. His remarks reflect the state of generative AI in fall 2023.
Why did generative AI take off so fast?
Franks compared the moment to the big data wave a decade earlier. Big data, he said, was a poor label because the real change was not volume but variety: text, images and web browsing history opened new use cases.
Earlier machine learning, he said, took a row of data and appended a prediction, forecast or classification. Generative AI instead uses historical data to generate an entirely new image or string of text. That, he argued, lets organizations solve problems that were not possible before.
What does generative AI look like in practice?
Franks pointed to a Wayfair tool in which a customer uploads a photo of a room, enters a prompt about color scheme, and sees a generated image featuring products that can actually be purchased. He said it combines traditional AI, generative AI and propensity modeling, and gets customers to options in minutes.
He did not see it threatening designers. They would rather spend time on full design clients than on ten quick mockups, nine of which will not buy anything. He also noted AI is being embedded so seamlessly, as in Adobe's tools, that users may not know they are using it.
Is "hallucination" the right way to think about it?
Franks called hallucinations the biggest misnomer around generative AI. A search engine matches keywords to real documents. A generative model, he said, reduces a prompt to a mathematical representation and produces a probable answer, so it is "effectively making up every single answer that it gives."
The surprise, in his view, is how often it is right. He asked the same question twice, a history of the world in 50 words, and got two different answers, neither clearly wrong. Users should expect that the model may not give the same answer a minute later, let alone a week later.
What does it get right and wrong?
Franks said the pattern is predictable. Recency matters, because the public models he cited were cut off around the end of 2021. Obscurity matters too: widely documented subjects like Pearl Harbor tend to come out right, while an obscure colonel in an obscure battle may be half made up.
He gave a personal example. The model said his first book had been nominated for an award by INFORMS, which he could find no record of. "If it can fool me about myself," he said, it can fool anyone on other topics.
He also cautioned against reading exam performance as general intelligence, since standardized tests mostly ask for recall of documented facts. And he said not to use it for math: "it's not a calculator." It predicts the most likely next string, which is usually right for 1 plus 1 but can fail on more complicated problems.
Where does it fit, and what are the risks?
Franks said it suits tasks where the general direction is enough. He cited a prototype he saw from Amazon that summarizes thousands of product reviews and tags themes such as ease of use. A summary like that can be generated once and shown to everyone.
For inventory counts or flight availability, he said, businesses need a lookup, not a probabilistic guess. He also raised model collapse: as generated content pollutes training data, models can drift toward the mean unless the data is filtered. On fair use, he offered a contrarian view that AI should not be held to a higher standard than human artists who learn from past work, noting the question will be in the courts for years.
His forecast was ensembles. Prompts would be parsed and routed to several language models, a math engine, search and domain plugins. The underlying models will be commodities, he said, so differentiation will come from the application layers companies build on top.
Where to learn more
The AI Enterprise Conference 2027 takes place Tuesday, August 31, 2027, at Pier Sixty in New York City, and COLLIDE 2027 returns to Sandy Springs Performing Arts Center in Atlanta on Wednesday, October 13, 2027. Details are at /events/ai-enterprise-conference/ and /events/collide/. Registration is coming soon.
Frequently asked questions
Why does generative AI hallucinate?
Franks said it does not match a prompt to real documents; it converts the prompt to a mathematical representation and generates a probable answer, so it is effectively making up each answer.
What does generative AI get right and wrong?
He said it does well on well-documented, consistent topics and poorly on recent events and obscure subjects. He also advised against using it for math or computation.
What is model collapse?
Franks described it as models trained on increasing amounts of AI-generated content drifting toward the mean and losing quality, which he said needs filtering to avoid.
How will companies differentiate with generative AI?
He predicted the underlying models will be commodities and that value will come from application layers that route parts of a prompt to models, search, math engines and plugins.
This article is based on remarks by Bill Franks, Director, Center for Data Science & Analytics, KSU at COLLIDE Data + AI Conference 2023 on October 3, 2023. More coverage: COLLIDE AI Conference.
Filed under: AI Use Cases & Applications, AI Governance & Trust
