Copilot Cooking Show: Building a GenAI Analytics Assistant (AnswerRocket)
At COLLIDE 2023, AnswerRocket CTO Mike Finley used a recorded demo to show how a generative AI copilot for analytics is taught new skills. A pasted JSON block and a short Python routine added e-commerce and cohort analysis, with numbers verified against the data.
By The AI Enterprise Magazine Team4 min read
Key takeaways
- Skills are tools. Finley described skills as functions the language model calls to fetch facts it does not have from enterprise data.
- Verified numbers. In the demo, figures turned blue once checked against data returned by queries, which he said guards against hallucination.
- Quick to extend. An e-commerce skill was added by pasting a JSON block, and a cohort analysis with a short Python routine.
- Context carries through. Max kept the May 2022 context across follow-up questions and switched metrics when the question changed.
- Reuse existing analytics. Finley said teams should bring their current algorithms into a copilot rather than change how they work overnight.
At COLLIDE 2023 in Atlanta, Mike Finley, CTO of AnswerRocket, used a "cooking show" format to demonstrate how an enterprise can build a generative AI assistant for data analysis. In a quickfire session, Finley showed a recorded demo of AnswerRocket's assistant, called Max, and added new analytical skills to it on the fly. He said the demo was recorded as a single take, in real time, because of uncertainty about conference Wi-Fi. This is a vendor demonstration, and the capabilities described are as AnswerRocket presented them in 2023.
What is Max, and what is it built from?
Finley described Max as an enterprise assistant, or copilot, that came out of about 10 years of AnswerRocket's work on augmented analytics, combined with roughly 10 months of integrating large language models. From the analytics side, he said, the company brings depth of analysis and enterprise-grade systems. From the language models it gets plain-language interaction, the ability to plan a multi-step analysis, and the ability to react to data fed to it.
Max connects to structured data and to what Finley called dark data: PDFs, meeting transcripts and annual reports that were not usable as data before language models. Outputs can go to chat, Slack, email, PowerPoint or PDF. Finley noted that customer concerns about data sharing, training competitors' models and hallucinated values are legitimate, and said AnswerRocket addresses them, but he did not cover them in the session.
What are "skills," and how do they work?
Skills, in Finley's explanation, are the tools a language model can call to reach outside its training data and get facts it does not have. He compared them to what OpenAI calls functions and LangChain calls agents or tools. Each skill encodes the data sets involved, the analysis techniques, the visualizations and the kinds of insights worth surfacing. As an example, he said an AI system once told him the top driver of COVID was the date, which is not useful.
AnswerRocket's Skill Studio is where a team teaches Max about its business. Finley called it the place where "we take Max to school."
What did the demo show?
Finley played the role of a marketing manager at a direct-to-consumer pasta company who wants to decide where to spend marketing dollars and find white space in the market. Max started with syndicated market data covering 30 months. Asked for last year's sales, it returned $2.3 billion and showed it had used a "pasta snapshot analysis" skill.
Numbers in the answer turned blue once Max had verified them against the data fed to the language model. Finley said this means they are not hallucinations, and later said they trace back to SQL statements. When he asked how much of a May spike came from e-commerce, Max said it had no channel-level data.
How was a new skill added?
Finley switched to the Skill Studio, where the pasta assistant had three skills: trending, snapshot and performance breakout. He added an e-commerce skill by pasting in a JSON block, which he compared to getting a skill from an app store. He then re-asked the question and Max used the new skill.
He went further, adding a cohort analysis. That meant updating the skill description and pasting a short Python routine into the analysis dispatch. Asked "did those customers stay with us for long," Max recognized the May 2022 context from the conversation thread, switched from revenue to customer counts and returned a cohort chart. Finley said the Q4 follow-up, asking which base sizes and segments drove sales, produced a table, commentary and a combined analysis of the two.
What does this mean for enterprises?
Finley's argument was that companies should not change how they forecast or analyze overnight. They should bring existing analysis and expert algorithms into a copilot to make them easier to reach and run at scale. He said a conversation thread could also be scheduled, for example run every morning for all product categories, and delivered to a phone or enterprise portal.
His closing point was that the assistant works on what a data set describes about the business, not just on time series or anomalies in the abstract.
Where to learn more
COLLIDE 2027 returns to Sandy Springs Performing Arts Center in Atlanta on Wednesday, October 13, 2027, and The AI Enterprise Conference 2027 takes place Tuesday, August 31, 2027, at Pier Sixty in New York City. Details are at /events/collide/ and /events/ai-enterprise-conference/. Registration is coming soon.
Frequently asked questions
What is a skill in an AI analytics copilot?
In Finley's demo, a skill is a tool the language model can call, much like a function in OpenAI's terminology, that retrieves facts or runs an analysis against enterprise data.
How does AnswerRocket's Max guard against hallucinations?
Finley said numbers in answers are checked against data fed to the model and shown in blue, and that they trace back to SQL statements.
How do you add a new analysis to the copilot?
In the demo, an e-commerce skill was added by pasting in a JSON block, and a cohort analysis by updating the skill description and adding a short Python routine.
What data can the copilot use?
Finley said it can use structured databases and unstructured sources such as PDFs, meeting transcripts and annual reports.
This article is based on remarks by Mike Finley, CTO, AnswerRocket at COLLIDE Data + AI Conference 2023 on October 3, 2023. More coverage: COLLIDE AI Conference.
Filed under: AI Use Cases & Applications, Observability & Data Quality
