Recommendations, search, RAG and AI agents share more than they seem. In a Data Science Connect webinar, Aerospike Chief Evangelist Sesh Seshadri explained how multi-step inference, feature stores and short-lived agent data sets shape real-time AI infrastructure, drawing on Myntra, PayPal and Barclays.
Key takeaways
- Deadlines are tight. Recommendations need about 50 ms, ad bidding 100 ms and search 200 ms, Seshadri said.
- Multi-step inference stretches the budget. Chained, cascaded and narrowing models all depend on fast feature lookups at each step.
- RAG is a search problem. Recommendations, search and RAG share the same retrieve-then-rerank pipeline and the same three inputs.
- Agents create micro data sets. Millions of short-lived, session-specific data sets must be written, read and expired quickly.
- Watch application and database metrics. Queries per second above plan or rising tail latency signal infrastructure strain.
Recommendation engines, enterprise search, retrieval-augmented generation and AI agents look like different problems. Underneath, they lean on the same kind of fast data layer, Aerospike Chief Evangelist Sesh Seshadri argued in a Data Science Connect webinar sponsored by Aerospike. In "Building High Performance Data Infrastructure for AI," hosted by DSC co-founder and president Brian Mink, Seshadri walked through how real-time data infrastructure supports predictive, generative and agentic AI, using architectures from customers including Myntra, PayPal and Barclays.
What does a real-time AI inference pipeline look like?
Seshadri used the production architecture of Myntra, which he described as India's largest fashion and lifestyle e-commerce store and part of Flipkart, as a template he said most training and inference pipelines follow. Data arrives from customer interactions and from catalog changes such as new products, prices and availability. A data platform splits it into batch ingestion into a data lake and real-time streaming into a state store that can absorb "a tsunami of writes."
An ML platform then trains models and generates offline features for batch jobs like email campaigns, while an online feature store serves real-time inference. Aerospike sits in three places in that design, he said: the state store, the online feature store and an inference cache.
Why do AI models need multi-step inference?
Business deadlines are tight. Seshadri said recommendations need to happen in 50 milliseconds, search in 200 milliseconds and real-time ad bidding in 100 milliseconds. Yet better models and more data produce better decisions. "More data beats clever algorithms," he said, calling it a running joke of his.
Teams reconcile those pressures with multi-step inference. Some chain models that generate signals stage by stage, such as a query's product category and price sensitivity, before a final model decides. Others cascade cheap models for easy cases and call expensive ones only when needed, as in fraud screening. Others start with a large candidate set and narrow it, like a recruiting pipeline that filters quickly before detailed interviews.
Each step needs a feature lookup, so lookup speed determines how much of the budget is left for modeling. If a feature store can return 500 candidates for reranking in under a millisecond instead of 100 in five, he said, the result is "more time for modeling, more modeling steps and more data to model."
Why are recommendations, search and RAG the same problem?
Seshadri said all three balance three inputs: the user's request and session intent, personalization from past behavior, and business context. In e-commerce, business context could mean pushing overstocked inventory or favoring higher-margin products. In enterprise RAG, it includes access controls over who can see which documents.
The pipeline is also the same. Retrieval narrows millions of documents to hundreds using vector search, keyword search or a database lookup, then a deeper reranking model picks the final few. "RAG is nothing but a search in the context of finding the right set of documents to feed to an LLM," he said. A feature store suited to recommendations is therefore suited to RAG, he argued.
For generative AI, the same layer stores conversation context and user history before a request reaches the LLM. Myntra's conversational shopping assistant and its customer support chat agent both keep conversation context in Aerospike, according to Seshadri.
What data does agentic AI need?
Predictive and generative systems are reactive, responding to requests, Seshadri said. Agents are proactive and goal-oriented. His example: an agent that continuously monitors recorded sales calls and flags customers who sound irate or likely to churn. An orchestration agent coordinates many smaller agents that form hypotheses, test them, act and study the results.
Across millions of users, those agents create huge numbers of small, short-lived, session-specific data sets that must be written, read and expired quickly. He called these "micro data sets" and said they fit a NoSQL data model.
How do you know your infrastructure can't keep up with AI?
Answering an audience question, Seshadri said real-time AI infrastructure differs from analytics pipelines on four counts: latency, throughput, cost and the ability to replicate fast-changing data from a core region to edge locations near users. Analytics queries scan terabytes or petabytes, while real-time requests touch kilobytes or megabytes.
To spot strain, watch both application and database metrics. If you provisioned for 10,000 queries per second at peak and see 12,000, or 99.9th-percentile response time drifts from 750 milliseconds toward one second, it's time to investigate. On the database side, he listed latency, throughput, uptime, data size and how the system handles losing a region during a traffic spike.
He also described how Barclays chose a database for card fraud detection after considering application caches, cache-as-a-service, in-memory databases and other designs. Its checklist included throughput, tolerance for latency variance, whether strong consistency is worth trading off some availability, hot and cold data, multi-cloud and on-premises deployment, and three-year total cost of ownership. Seshadri said Barclays saw four times the throughput and that Wayfair went from 60 servers to seven after replacing Cassandra.
Explore AI infrastructure on DSC webinars
Data Science Connect has 24 free live webinars on its 2027 schedule across four series: the AI × Enterprise Panel Series, AI Demo Days, Agentic AI Showcases and the Plug & Play Series. See the full lineup on the webinars page.
Frequently asked questions
What is multi-step inference in machine learning?
Splitting a real-time decision across several models, for example running cheap models on easy cases and expensive ones only when needed, so the best possible decision fits within a fixed time budget.
How is RAG similar to a recommendation system?
Seshadri said both retrieve hundreds of candidates from millions of documents using the request, personalization and business context, then rerank them; RAG is search for the right documents to feed an LLM.
How does real-time AI infrastructure differ from analytics infrastructure?
Real-time systems prioritize latency, throughput, cost and fast replication to edge regions, and touch kilobytes or megabytes per request, while analytics queries scan terabytes or petabytes.
What data infrastructure do AI agents need?
Fast storage for conversation context and for many short-lived, session-specific micro data sets that agents create and discard, according to Seshadri.
Watch: Building High Performance Data Infrastructure for AI on YouTube
Event: Webinars · More coverage: Webinars insights
