There’s a problem facing today’s enterprises seeking to scale AI: The capacity to support that demand just doesn’t exist. So how does the market adjust?
Vultr CEO J.J. Kardwell took the stage at a HumanX panel last week in San Francisco to address, among others, that question. He talked about how the AI industry has reached a point where supply doesn’t match demand, how the industry can correct course, which businesses will survive the memory shortage, and how independent clouds are well-suited to a moment when efficiency is paramount.
How did we get here – and how do we fix it?
Kardwell’s analysis of how AI infrastructure came to be so underbuilt is straightforward: Until recently, the buy-in fell short.
"A year ago, as a market, we were not collectively convicted and committed to the impact of AI. That’s the reality,” he said. "So buyers were not yet at a place where they were willing to commit on a long-term basis, right? Very much a wait-and-see mode.”
Of course, everything has changed. Having seen real-world evidence of the breakthroughs that AI can offer the enterprise, buyers have done a full 180-degree pivot. As Kardwell pointed out, companies like Anthropic are making that clear with astronomical growth rates.
The rush to adopt AI has happened at a pace that the industry wasn’t prepared for. The way to solve that problem is not to repeat that same mistake. It’s to commit more deeply.
“The only way out of it is a deeper commitment to longer-term planning,” Kardwell said. "And we’re seeing that already in the market. Customers are making five- and six-year commitments to infrastructure, whereas even nine months ago, people were reluctant to commit for more than one or two years. And that’s what it takes."
Who will make it – and who won’t?
Hardware prices are rising for providers. "The best numbers I’ve seen suggest a 30% gap between supply and demand,” Kardwell said. It’s a clear sign that capacity is short.
Limited capacity means limited investment. There’s a finite amount of memory that the market can support (and will be able to support for the short-term future). Buyers and investors will need to be selective about which applications are worthy of that limited capacity.
“The market will favor allocating capacity to the highest and best uses,” Kardwell said. "And that means those companies that have more strained business models will really struggle to justify the investments."
That may be a positive. With more impactful AI applications placed at a premium, the market will be populated by increasingly high-value, high-impact use cases.
"We’re talking about a market competing for what is effectively a fixed amount of capacity over the next two years,” Kardwell said. "And in some ways, that’ll accelerate the maturation of the market.”
However, Kardwell said the capacity shortage is inspiring two other trends. One is buyers and investors emphasizing optimal utilization to avoid procuring reserved capacity that is often underused.
Another is a resurgence of previous-generation hardware, revitalizing a broad array of hardware that might have once been passed over.
“We’re seeing, at an innovation level, an effort to get more out of less resource,” Kardwell said. “We’ve seen a resurgence in demand for gear that’s two to three generations older. Even going back to [NVIDIA] A100, [NVIDIA] H100, and MI300X on the AMD side. Their models are being optimized to make use of that older gear.”
Kardwell compared it to the automobile industry just a few years ago. When supply chains were constrained during COVID, demand for used cars grew. We’re seeing a similar trend with AI infrastructure.
A new era with new priorities
When capacity is low, timelines are strict, and impact is necessary, efficiency is everything. That’s why infrastructure that worked years ago doesn’t necessarily work today.
“Many of the things that were advantages in the pre-AI infrastructure world,” Kardwell said, "in many ways are disadvantages now.”
For example, Kardwell pointed out that the upstack features that drove – and still drive – lock-in on hyperscaler platforms, like Platform-as-a-Service and Software-as-a-Service offerings, are less valuable today. Builders are embracing container orchestration and abstracting their own platforms from the infrastructure, preventing hyperscaler platforms from trapping them.
Kardwell sees these developments as reflective of a shift in market share away from hyperscalers and toward independent cloud providers.
“Efficient delivery, customer experience matter more than ever,” Kardwell said. “Reliability obviously is foundational. Trust, security, privacy – foundational, things we think a lot about and have for the 24 years Vultr has been in business. But, ultimately, this is a world where efficiency matters. And independents have an enormous advantage.”