Why Enterprise AI Inference is Driving Decentralized Infrastructure
As enterprises look to operationalize and scale agentic AI, the technology debate is shifting from centralized training at scale to local and decentralized inference. A full-stack approach to enterprise AI accelerates the journey.
Back in the early days of computing, everything was on a massive scale. The first electronic mainframes filled entire rooms, their vast control panels an expansive suite of dials, knobs, and levers.
Today, of course, computers can be as small as a watch, and most office tasks are conducted on streamlined laptops. The cloud has removed much of the need for on-premise data centers. Where they are still required, the real-estate footprint is much smaller than in the past.
This same shift – from the fixed and centralized to the flexible and dispersed – is currently underway at the cutting edge of enterprise technology: AI.
From the moment enterprise-grade AI first hit the big time (just three years ago, although it feels much longer), the conversation focused on hyperscale cloud infrastructure and frontier-lab innovation.
That makes sense. The frontier labs have been at the forefront of innovation, and their work – building and training ever-larger LLMs – requires massive data-center capacity and hyperscale GPU clusters.
As in the early days of computing, everything has been large scale and fixed. Just as mainframes couldn’t be taken out of the rooms in which they were built, nor could closed, proprietary LLMs run anywhere except on the lab’s cloud infrastructure.
Enterprise AI changes all this. As AI becomes integrated into the daily operations of businesses, AI models and infrastructure are starting to change too. AI “in the open” is a very different beast. Successful enterprises understand these key differences and adapt their infrastructure architecture and software stack accordingly.
What AI “in the open” means
The concept of AI “in the open” is simply a recognition that AI is moving from labs to enterprise systems. But it’s also a recognition that as the shift takes place, the cloud infrastructure and software that enterprises use to support AI will increasingly be open, leveraging open standards and technologies.
In addition to the closed models in use today, enterprises will also increasingly adopt open AI models. Although closed models from heavily capitalized frontier labs will most likely maintain an advantage in performance and innovation, open models enable businesses to fine tune model weights on internal corporate data, tailoring them much more closely to their business needs. Additionally, costs can be more predictable, as hosting open models on dedicated infrastructure can give enterprises more control over spend.
Openness is also important from an infrastructure and software perspective. Our annual AI maturity report found that avoiding vendor lock-in was the top-cited reason for shifting AI workloads between venues, and 37% of respondents say partnering with alternative cloud providers is their main strategy for building new AI use cases.
The motivation is easy to see. The infrastructure provided by legacy hyperscalers can leave organizations locked into their particular “walled garden,” with proprietary APIs and heavily managed AI pipelines binding the enterprise to the ecosystem. Meanwhile, service bundling means that businesses can end up paying for AI services they don’t actually use, unable to integrate technologies that might better serve their specific use cases.
Finally, running AI “in the open” means bringing compute closer to where workloads run. This is because unlike traditional LLM prompts that involve a single request-and-response cycle, autonomous agents operate in dynamic, multi-step, and continuous feedback loops. Local compute is therefore vital to reduce latency and to keep sensitive data within the protected enterprise perimeter.
As businesses rewire their workflows for agent execution, they therefore need a different approach designed to optimize multi-agent orchestration, enable continuous machine-scale execution, and deliver compute to where workloads run. That means embedding simplicity, flexibility, and openness into the entire AI stack.
What does agentic AI-ready infrastructure look like?
Cloud infrastructure is ideally suited for helping businesses evolve their enterprise systems for the agentic era. It removes the time and effort required to manage complex technology integrations and provides a simple way to access the full range of CPU and GPU hardware needed for complex agentic workflows.
However, this cannot be the proprietary approach of the past. Businesses require flexible infrastructure that they can tailor to their unique needs, local resources to meet their data privacy and sovereignty requirements, and infrastructure that helps them make the economics stack up.
Vultr believes that the best way to achieve these aims is to deploy integrated, heterogeneous, full-stack AI infrastructure and software. To be fit for purpose in the age of agentic AI, this requires:
- CPU + GPU architecture. Vultr supports AMD EPYC™processors for CPU workloads and AMD Instinct™ processors for GPU requirements. The GPUs take care of AI reasoning, while the CPUs deliver agentic orchestration and data pipelines, agentic sandbox capacity for secure execution, and capacity to support the surge in demand on core cloud workloads that AI agents drive. Vultr VX1™ is Vultr’s CPU proposition. As well as providing the high performing CPU capabilities enterprises require, it also delivers a strong performance-to-cost ratio. When used for commodity cloud compute, Vultr VX1 can therefore release budget to invent in agentic CPU and GPU infrastructure.
- An agentic-ready data foundation. Vultr integrates VAST Data’s AI operating system for storing, managing, accessing, and processing data, providing the AI-ready data infrastructure needed to keep compute productive, data accessible, and demanding AI workloads moving efficiently from development to production, ensuring that enterprises get maximum value from their investments.
- A standardized software stack. Vultr’s full-stack approach also integrates the SUSE AI Factory, an AI software layer that extends SUSE Linux and Rancher Prime with validated AI blueprints, GitOps automation, and governance. This creates a standardized AI platform that supports the full AI lifecycle from development to production across cloud, edge, on-premises, and sovereign environments. SUSE helps accelerate AI pilots to production through validated blueprints, lifecycle management, governance, and scalable infrastructure.
Full-stack AI infrastructure makes mastering AI in the open simple and scalable, while helping to make the economics work. It’s an ideal model for the emerging AI landscape where inference is increasingly decentralized and local. It puts enterprises in control of their own destinies, giving them the freedom to build AI systems on their own terms while still benefitting from the simplicity of the cloud. It’s the way to go for any businesses looking to operationalize AI rapidly and securely.
To learn more about scaling AI through full-stack infrastructure, download our new whitepaper: “Mastering AI in the Open: Vultr Full-stack Infrastructure for Enterprise Agentic AI.”
