In a Data Science Connect webinar with HPE and NVIDIA, three technologists explained the AI factory model for enterprise AI infrastructure. They covered packaged, turnkey and custom deployments, why cooling now shapes data center design, and the strategy and skills gaps that leave GPUs sitting idle.
Key takeaways
- An AI factory is a relay team. Shirhatti said compute, networking, software and ecosystem only win when a reference architecture makes the handoffs clean.
- Three deployment paths. Heibein described packaged AI applications, a turnkey private cloud AI in t-shirt sizes and custom AI factories.
- Cooling is now a design constraint. Rising chip power and falling case temperatures push large systems toward direct liquid cooling.
- GPUs without strategy sit idle. Guddad said some customers have hardware installed but no defined use cases or outcomes.
- Focus on the end user. Employees expect the accuracy, speed and experience of consumer AI tools.
An AI factory takes in electricity and data and produces tokens, intelligence and business outcomes, Kaushik Shirhatti, VP of Global AI Networking at NVIDIA, told attendees of a Data Science Connect webinar brought to the audience by Hewlett Packard Enterprise and NVIDIA. He was joined by two HPE leaders: Steve Heibein, Federal AI Chief Technologist, and Bhuvaneshwari Guddad, Chief Technologist for Worldwide AI Services. Brian Mink, co-founder and president of Data Science Connect, moderated.
The session explained what an AI factory is, the deployment options HPE and NVIDIA offer, and the strategy, skills and data challenges that slow enterprises down. "Easy is hard," Shirhatti said. Making AI look seamless for users takes a great deal of work behind the scenes.
What is an AI factory?
Shirhatti compared an AI factory to a 4x100-meter relay team. Four fast sprinters aren't enough; teams win on clean handoffs. In his analogy, the sprinters are compute, which he called the heart of the factory; networking, the backbone; software, the operating system; and the ecosystem of partners and developers. The handoffs are governed by a reference architecture that optimizes the full stack for a given workload.
What matters to end users, he said, is not how many engineers it took to stand up a GPU cluster. It is "time to first token," time to first inference and, ultimately, "time to wow and time to value."
He described three phases organizations go through: prioritizing among many AI ideas, getting the data ready, and choosing infrastructure. For infrastructure, enterprises can rent from a large cloud provider, rent from a specialized AI cloud, or build their own, which means planning for power, cooling, GPUs, networking and certified systems.
What are the options for deploying an AI factory?
Heibein outlined three paths to AI productivity:
- Packaged AI applications, such as video surveillance analytics, fraud detection or shopping recommendations. These require configuration, not code, and the software dictates how many servers and GPUs are needed. He said NVIDIA's catalog includes more than 1,500 such applications.
- A turnkey AI factory, HPE Private Cloud AI, co-developed with NVIDIA. It uses what Heibein called an "opinionated software stack" and comes in t-shirt sizes, from small to extra large with four or eight GPUs per server, plus a developer system for teams that don't yet know which size they need.
- A custom AI factory, built to the customer's requirements on a proven reference architecture, either from scratch or fitted around an existing HPC or AI software stack.
Private Cloud AI arrives racked and cabled and is usually operating within 24 hours of reaching the data center floor, Heibein said. It includes NVIDIA AI Enterprise software, NVIDIA NIM inference microservices, HPE AI Essentials as the user interface and a no-code, low-code retrieval-augmented generation tool for building chatbots on a company's own content. He said it gives teams a cloud experience on premises at a fraction of cloud costs, which he said "can get out of control."
Why does cooling matter for AI infrastructure?
Heibein said GPU and CPU power keep rising while the case temperature chips need to stay at keeps falling, partly because chips are now stacked in several layers. At some point air cooling no longer works. Direct liquid cooling puts fluid right on top of the chip. For data centers without liquid cooling, HPE offers modular data centers, air or liquid cooled, that can be brought on site.
He added that large AI factories should support multi-tenancy, GPUs as a service, self-service provisioning and automated billing, so the investment can serve many teams across an organization.
What challenges keep enterprises from scaling AI?
Guddad said she had just left a customer workshop where the company had plenty of GPUs installed but didn't know which use cases to run on them. Strategy and readiness come first, she said, with outcomes tied to each business unit so projects don't stay siloed.
Other obstacles she listed were in-house expertise, data quality ("garbage in is garbage out"), a fast-changing tool chain with complex version dependencies, inference cost optimization and compliance rules that differ by geography.
HPE structures its services from "day minus one" to "day two," she said: advisory and use-case definition, then design and planning, implementation and integration, and finally running and optimizing with MLOps. Data and security services, including risk assessment and guardrails against prompt injection, run alongside.
How are AI factories being used across industries?
Guddad said priorities vary by sector. Finance and healthcare put compliance and data security first, while manufacturing and retail focus on handling data at the edge. Use cases she cited included fraud detection and credit scoring in finance, genome sequencing and drug discovery in healthcare, and predictive maintenance and computer vision in manufacturing.
Heibein said federal agencies and systems integrators often build centers of excellence on AI factories to consolidate resources, avoid shadow IT and keep control of intellectual property. "It's proven technology," he said.
Shirhatti's advice was to "maniacally focus on prioritization of the projects" and on users, who expect the accuracy, speed and experience they get from consumer AI tools at home.
More AI infrastructure discussions in Data Science Connect webinars
Data Science Connect's 2027 schedule includes 24 free live webinars across four series: the AI × Enterprise Panel Series, AI Demo Days, Agentic AI Showcases and the Plug & Play Series. Sessions bring infrastructure vendors and enterprise practitioners together on scaling AI. See the webinar schedule.
Frequently asked questions
What is an AI factory?
Shirhatti described it as infrastructure that takes in electricity and data and produces tokens, intelligence and business outcomes, built from integrated compute, networking, software and partners.
What is HPE Private Cloud AI?
According to Heibein, it is a turnkey on-premises AI platform co-developed with NVIDIA, delivered racked and cabled in several sizes and usually running within 24 hours of delivery.
Why do AI data centers need liquid cooling?
Heibein said chip power keeps rising while the temperature chips must stay at keeps falling, so beyond a certain point air cooling can't remove the heat and fluid must cool the chip directly.
What are the biggest barriers to scaling enterprise AI?
Guddad listed strategy and readiness, in-house skills, data quality, tool chain complexity, inference cost and regional compliance requirements.
Watch: AI Factories at Scale Unlocking Innovation with NVIDIA and HPE on YouTube
Event: Webinars · More coverage: Webinars insights
