AI Workstations vs Cloud Costs: Lenovo's Case at COLLIDE 2023
In a session sponsored by Lenovo at COLLIDE 2023 in Atlanta, Mike Leach argued that AI teams should match hardware to the workload and treat workstations as an on-ramp to the data center. He also said IT, data science and finance need to plan AI spending together.
By The AI Enterprise Magazine Team3 min read
Key takeaways
- Start with the problem. Leach said AI should be tied to a pain point or inefficiency, not pushed where it does not fit.
- Most data science work is development. He split AI computing roughly 80% development and 20% integrated tools, and said laptops are often underpowered for the former.
- Workstations as an on-ramp. He argued local workstations let teams test and fail cheaply before scaling to cloud or servers, and cited savings above 80% among Lenovo's top accounts.
- Cloud bills can balloon. A retail example had interns spending about three-quarters of a million dollars in the cloud.
- Align IT, data science and finance. Leach said these groups often have conflicting priorities and should plan AI infrastructure together.
At COLLIDE 2023 in Atlanta, in a session sponsored by Lenovo, Mike Leach, Senior Manager for Workstation and Vertical at Lenovo, argued that the computing hardware behind AI work deserves as much attention as the models. Leach, who described Lenovo as a gold sponsor, said he did not want the 17-minute talk to be a sales pitch. His claims below reflect a hardware vendor's perspective.
Why does AI need a business case before hardware?
Leach said AI is "a journey and not a destination," a new stream of business that needs resources, budget and buy-in at the highest level. He said chief technology officers were "writing blank checks" to bring AI into their businesses, and that those checks "don't last forever."
His advice was to start from a problem, such as an inefficiency or pain point, and work back to where AI can help, rather than forcing AI into places it does not fit. That approach, he said, avoids the proof of concept that "never goes anywhere" and only spends money.
He also asked the audience to place themselves on a four-stage scale: stage zero with no plans, stages one and two using open source or cloud-provider tools or building in-house on proprietary data, and stage three deploying AI at scale.
What does the computing landscape for AI look like?
Leach described computing for AI as an iceberg, with much of it hidden. He split the market into roughly 20% integrated AI, such as Microsoft's Copilot-style tools, and 80% development work, where teams use heavy compute to prepare data and build the intelligence. Corporate-issued laptops, he said, are rarely suited to the second category, and IT decision-makers often do not understand the software data scientists use.
Citing the Anaconda State of Data Science report, Leach said a workstation can handle 85% of a data scientist's day-to-day workflow, including data preparation, cleansing, visualization and model selection.
When does a workstation make more sense than the cloud?
Leach positioned the workstation as the middle option between a PC and the cloud: more performance than a laptop at lower cost than cloud service providers. "There is no such thing as the cloud," he said. "It is just somebody else's computer."
He argued that once teams are "hooked" on the cloud, token and compute costs can become prohibitive at scale. His suggestion was to treat a workstation as an on-ramp to the data center, a place to test quickly, work on local data, and "fail fast and fail cheap" before scaling to servers or the cloud. Leach said Lenovo's analysis of its top 100 global accounts showed savings in excess of 80% when teams developed on mobile workstations and used the cloud to augment, rather than replace, that work.
He added that organizations working with sensitive data, such as a hospital that cannot take data outside its network, could run a generative AI chatbot locally on a workstation or edge device.
Why are IT, data science and finance out of sync on AI spending?
Leach described conflicting priorities. IT decision-makers, he said, work with small budgets, a restricted catalog and a focus on security, and are often not at the table for AI strategy. Data science teams, by contrast, have near-unrestricted budgets and may buy hardware outside the catalog and rely on the cloud. The CTO writes blank checks, the CFO tries to save money, and the cloud bill grows.
He cited a retail customer whose two summer-intern data scientists, working on basic laptops, spent about three-quarters of a million dollars with a cloud provider. A workstation purchase of $50,000 to $60,000 had been rejected as too expensive, Leach said, so the business spent far more overall.
His conclusion was that IT, data science and finance should be in the room together. He also mentioned device-as-a-service leasing as a way to turn capital spending into operating expense.
Where to hear more on enterprise AI infrastructure
COLLIDE 2027, billed as the largest AI conference in the Southern U.S. with more than 1,000 enterprise data and AI leaders, takes place Wednesday, October 13, 2027, at the Sandy Springs Performing Arts Center in Atlanta. Details are on the COLLIDE event page, and registration is coming soon.
Frequently asked questions
Is it cheaper to run AI development on a workstation or in the cloud?
Leach, a Lenovo executive, argued that developing locally on workstations and using the cloud to scale can cut costs, citing savings above 80% among Lenovo's top 100 global accounts. The figure is a vendor claim.
What is a workstation for data science?
Leach described it as a higher-powered computer between a standard PC and the cloud, suited to data preparation, visualization, model selection and training on larger data sets.
Why do AI cloud costs grow so quickly?
He said teams that rely on cloud resources with unrestricted budgets can run up large bills, citing a retail case of about three-quarters of a million dollars from two interns.
Who should be involved in AI infrastructure decisions?
Leach said IT decision-makers, data science teams, the CTO and the CFO should all be part of the AI strategy conversation.
This article is based on remarks by Mike Leach, Senior Manager, Workstation & Vertical, Lenovo at COLLIDE Data + AI Conference 2023 on October 3, 2023. More coverage: COLLIDE AI Conference.
Filed under: Data & AI Infrastructure, AI Strategy & ROI
