Vector Search, RAG and Data Platform Convergence: Redis at COLLIDE
At COLLIDE 2023 in Atlanta, Redis Field CTO Allen Terleto argued that data platforms are converging and that vector search is the newest capability vendors are adding. He explained how RAG and semantic caching can reduce LLM cost, latency and data risk, and what enterprises should weigh when choosing a vector database.
By The AI Enterprise Magazine Team4 min read
Key takeaways
- Platforms are converging. Terleto said competition and growth pressure pushed database, streaming and search vendors to overlap, and vector search is the newest addition.
- Vectors make unstructured data searchable. Models encode audio, video and documents into numeric arrays that can be matched by similarity in milliseconds.
- Check the cache first. A semantic cache can recognize repeated questions with different wording and skip a costly LLM call.
- RAG beats fine-tuning for proprietary data. Retrieved context adds institutional knowledge without retraining, data leakage or slow responses, he said.
- Enterprise fit matters. Disaster recovery, Kubernetes support, managed services, skills and vendor count weigh as much as features.
At COLLIDE 2023 in Atlanta, Allen Terleto, Field Chief Technology Officer at Redis, argued that the data platform market was converging and that vector search was the latest capability vendors were folding into existing products. In a 20-minute session, he explained how retrieval augmented generation (RAG) and semantic caching could cut the cost and risk of using large language models in the enterprise.
Terleto said he had worked in the in-memory database and NoSQL space for more than a decade and had watched the market move through several hype cycles. Generative AI, he said, was the newest one, and vector databases were at the center of it.
Why are data platforms converging?
Terleto traced the history. Early databases, including mainframe and Oracle systems, were built when storage was expensive, so reducing storage cost came first and performance second. When memory costs fell in the 2000s, he said, distributed databases could "prioritize data access" and performance instead, which produced categories such as document stores, key-value stores, event streaming and search.
Then, he said, competition and cheap venture capital pushed vendors to keep growing after they had saturated their core use cases. The result was overlap: an event-streaming company positioning itself as a database, document databases moving into stream processing, and relational databases adding JSON support. Terleto said the downturn in macroeconomic conditions and tighter enterprise budgets were reinforcing the trend.
What is a vector database and how does vector search work?
Terleto described vectors as a way to search audio, video, PDFs and other unstructured data. A machine learning model encodes a piece of content into a multi-dimensional array of numbers, which he said could shrink a 10 GB video to a few kilobytes. Systems can then use measures such as cosine similarity to find the closest matches in milliseconds.
His retail example: click on the pants in a photo, find visually similar items in the database, and return purchase information. He said Redis had supported secondary in-memory indexing, full-text search and a SQL interface for years, and had been working on vector similarity search for about two years at the time.
Why is using LLMs on their own expensive and risky?
Terleto listed several problems with calling a foundation model for every request. Inference carries a GPU cost each time. Repeated questions get paid for repeatedly, and irrelevant questions waste money. "LLMs don't have any kind of memory," he said, "so they need a vector database or something to actually store that data."
Fine-tuning a model on proprietary data brought its own issues, he said: training cost, potential data leakage, the need to retrain as data changes, and slow responses. He also noted that models can give different answers to the same question and may hallucinate.
How does RAG with semantic caching work?
Terleto walked through a simplified flow. A user's prompt is encoded as a vector embedding. "We don't want to make the call to the LLM right away," he said. "We want to say, did we answer this already?"
He used a New York taxi analogy. A driver asked to go "to the city" and to Manhattan would give the same answer, and semantic search can recognize that equivalence where a conventional database would not. A cache hit avoids inference cost and responds in sub-milliseconds.
On a cache miss, the system checks whether the question is relevant at all, using semantic search as a guardrail. If it is, the vector database retrieves the top matching answers and document passages, adds session chat history, and sends that context to the LLM with the prompt. Terleto said this brings proprietary information into the answer without fine-tuning, improves accuracy, and reduces cost and latency.
How should enterprises choose a vector database?
Terleto said new vector databases appeared nearly every month and predicted both winners and losers. For enterprises, he said, the questions go beyond features: disaster recovery or active-active continuity, Kubernetes and OpenShift support, a managed service controllable through infrastructure as code, and ecosystem integrations.
He also pointed to people costs, including developer skills, training, onboarding, hiring and a vendor management process that can take three months. With companies trying to reduce vendor counts, he said, that favors converged platforms. Convergence around vector search, he noted, was happening "within months" rather than over ten years.
His claims reflect the perspective of a vendor that sells a converged platform.
Where to follow data and AI infrastructure discussions
Sessions like this one are part of what COLLIDE brings to Atlanta each fall. COLLIDE 2027, billed as the largest AI conference in the Southern U.S. with more than 1,000 enterprise data and AI leaders, takes place Wednesday, October 13, 2027, at the Sandy Springs Performing Arts Center in Atlanta. Details are at COLLIDE event page, and registration is coming soon.
Frequently asked questions
What is data platform convergence?
Terleto described it as purpose-built platforms such as databases, event-streaming and search products adding each other's capabilities as competition and growth pressure pushed vendors beyond their original categories.
What is a vector database used for?
It stores numeric embeddings of text, audio, video or images so systems can find similar items quickly, for example matching a product in a photo or retrieving relevant documents for an LLM.
How does semantic caching reduce LLM costs?
The system encodes each prompt and checks whether a similar question was already answered. On a hit it returns the stored answer instead of paying for another inference call.
Why use RAG instead of fine-tuning an LLM?
Terleto said fine-tuning is costly, can leak data, must be repeated as data changes and can be slow. RAG supplies retrieved proprietary context at query time.
This article is based on remarks by Allen Terleto, Field Chief Technology Officer, Redis at COLLIDE Data + AI Conference 2023 on October 3, 2023. More coverage: COLLIDE AI Conference.
Filed under: Data & AI Infrastructure, AI Use Cases & Applications
