Windows Server 2028 preview brings AI inference on-premises
Microsoft’s Windows Server 2028 public preview targets enterprise AI inference with local accelerators, hardened defaults and model support spanning Foundry, vLLM and Llama.cpp.

Why it matters
- Microsoft says Windows Server 2028 is in public preview and is its largest platform update since Windows Server 2025.
- Microsoft plans AMD Instinct MI350P preview support for the first half of 2027.
- Microsoft says Server Core enables hundreds of security settings on first boot by default.
- Microsoft’s support schedule runs through 2034 for Windows Server 2025 and Oct. 14, 2031 for Windows Server 2022.
- John Annand says local AI can require Nvidia or AMD accelerators, more RAM and 50GbE or 100GbE+ networking.
By Elena Vasquez
Microsoft’s Windows Server 2028 is now in public preview, with AMD’s Instinct MI350P card slated for a preview in the first half of 2027. Microsoft calls it the largest platform update since Windows Server 2025.
Microsoft says the release brings enterprise AI inference into customer-controlled infrastructure, close to physical operations. That matters because enterprises weigh cost, latency, intellectual-property protection, data residency and regulatory duties before they choose where models run, Windows Server product team leader Neil Hinnant said.
At Microsoft Ignite this week, Microsoft said Server 2028 supports “enterprise hybrid intelligence.” The preview brings AI accelerators, storage tools for accelerated data transfer, security controls meant to establish a “secure baseline from day one,” and simpler administration dashboards.
Microsoft also promises gains across automation, management, identity, security, recovery, storage and virtualization. Many features remain at different stages of availability and maturity.
Infotech associate VP John Annand put the caveat plainly: “It’s important to remember that this is the public preview phase, so a lot still remains a promise on the roadmap,” he said.
Where can enterprise AI inference run?
Microsoft positions Windows Server 2028 as a foundation for “enterprise hybrid intelligence,” designed to connect on-premises inference with Microsoft Foundry and other Azure cloud services.
Hinnant said Server 2028 can run inference close to physical operations and inside customer-controlled ecosystems. Customers do not have to build and operate a “separate AI stack for every location,” he explained.
“As AI-accelerated workloads diffuse across the enterprise, different workloads have different needs,” Hinnant wrote. “Cost, latency, IP protection, data residency, and regulatory requirements all shape that decision.”
Microsoft is introducing AI accelerators in its servers. AMD’s Instinct MI350P card will enter preview in the first half of 2027 for supported Windows Server systems, including Server 2028. The company is also developing tools to manage that hardware, deploy and configure inference environments, and direct model requests to appropriate environments.
Microsoft says the offering also begins its long-term investments to extend Microsoft Foundry to “fleets of local token factories.” Microsoft also plans support for open-weight and custom models across Foundry Local, Windows ML, vLLM and Llama.cpp.
What changes in the secure platform?
Microsoft says Server 2028 has a hardened Server Core configuration “by default.” Hundreds of security settings switch on during first boot, and Microsoft says the defaults align with industry benchmarks. “The secure configuration is the starting state, not a project you run after setup,” Microsoft explained. Enterprises need only relax the settings their workloads require.
Microsoft says that approach “closes the window where a newly deployed server sits exposed while it waits to be hardened.” Microsoft says the server also adds a containment layer for isolation, Agent 365 for agent identity and governance, and virtual machine protection that builds a chain of trust. It is investing in automatic certificate management to verify human and machine identities, prevent compromise and encrypt traffic.
Microsoft is also adding post-quantum cryptography. The company says the controls balance day-to-day security work with longer-term modernization “against the threats of tomorrow.”
Microsoft says Windows Server 2028 also supports native non-volatile memory express (NVMe) storage by default. The native support will “give demanding workloads more room to grow.”
How do administrators manage Windows Server 2028?
Microsoft’s platform also targets the need to apply policy, manage updates and inspect what runs across large estates. Intune management for Windows Servers is now in preview. It brings configuration, compliance and update workflows to eligible and always-on servers.
Windows Admin Center Virtualization Mode (WAC vMode) is now generally available on Windows Servers. Hinnant said it lets administrators work with specific servers and infrastructure “as one fabric.” “When a workload needs attention or a virtualization host needs troubleshooting, administrators need to get close to the machine and understand what is happening,” he said.
Microsoft limits the hybrid intelligence features to a limited number of customers through an early-access program with hardware requirements. Interested customers can apply through the Windows Server Technology Adoption Program (TAP). NVMe and Intune management for servers are available for testing now.
What will local AI cost enterprises?
Annand said customers running local AI on existing hardware would have to add Nvidia or AMD accelerators. He noted a lot of interest in AMD’s ROCm open-source GPU software. Those systems would also need “lots and lots of ever more expensive RAM,” he said. The upgrade can extend beyond servers into the network, too.
Annand said 10GbE network cards that were “blazingly fast” 3 years ago will likely bottleneck AI workloads. He said customers will probably need to upgrade to 50 or 100GbE+ to avoid that limit.
“To the surprise of no one, if you want to use Server 2028 for AI, you’ll need to invest in a new kit,” Annand said.
Many enterprises care more about quality-of-life improvements than AI hype around platform scaffolding, Annand said. For those customers, Server 2028 offers a “viable midpoint upgrade for existing hardware.”
Annand described Microsoft’s value proposition as ecosystem and product bundling, alongside enterprise control. He said Server 2028’s AI power will come from its integration with Microsoft Foundry and other Azure cloud services. Local inference remains a hot-button topic because security and cost control matter to Microsoft’s target customers.
Running AI in the public cloud has proven expensive, Annand said. He sees the Server 2028 feature set as a “tacit acknowledgement that some Microsoft customers want to return to on-prem from the blue skies of their Azure environments.”
Is Windows Server 2028 ready for production?
Microsoft says Server 2028 remains a prerelease product as it moves toward general availability. Some elements may be “substantially modified” before the official release.
Locked-down security and memory defaults can also disrupt software that works on Windows Server 2025. Microsoft identifies older protocols, legacy authentication and some management tools and scripts as possible problems. The company advises enterprises to test applications and operational tooling against Server 2028’s configurations before deployment. They should relax settings only when required.
Annand said not every enterprise will need an immediate upgrade. Microsoft’s support schedule keeps Windows Server 2025 under extended support until 2034. Previous editions remain covered through Windows Long-Term Servicing Channel (LTSC): Windows Server 2022 until Oct 14, 2031; Windows Server 2019 until Jan. 9, 2029; and Windows Server 2016 until its January 12, 2027 sunset.
Formerly known as the Long-Term Servicing Branch (LTSB), LTSC provides a stable, feature-locked operating system. Microsoft says the channel suits mission-critical, highly regulated and restricted environments. Buyers can therefore defer Server 2028 while they assess compatibility, infrastructure costs and practical AI use cases.
Can late adopters get ahead?
Annand said many AI pioneers do not wait for Microsoft to make a product enterprise-safe or ready. They spin up their own Linux clusters, work with Neocloud providers, or use AWS Bedrock and Azure Foundry. Waiting lets them solve infrastructure problems earlier, but at greater cost and technical risk.
“One strategy is to let everyone else pay the highest prices and experience the most technical difficulties when something first comes out,” Annand said. Later adopters may have a shorter path from investment to profitability, he said, while pioneers continue searching for it.
Annand expects Server 2028 to serve a late-adopter group once enterprise AI becomes more settled. “Enterprise AI on Server 2028 will eventually be an option for the fourth wave of AI adopters,” who are “the extremely cautious ones,” he said.
Microsoft’s long-term bet is that enterprise inference will spread across local “fleets of local token factories” while Foundry and Azure provide the wider ecosystem. The near-term decision is narrower. Enterprises must test preview software, budget for new hardware and decide which stricter defaults their workloads can safely change.
Source: networkworld.com
More from Elena Vasquez
Show full bio
Market editor covering technology and product news in artificial intelligence and machine learning at AI In Context.
744 articles
Related articles
- AI Agents Join the Boardroom: 2027 Forecast Says Workforce Planning Will Never Be the Same
- IBM and Microsoft Launch Imagine Enterprise for Industry AI Agents
- AI Compute in 2027: Serving Economics Will Reshape Data Centers
- Only 10% of IT Chiefs Use Agentic AI for Modernization, Study Finds
- Hitachi Vantara pushes AI analytics into storage with VSP One update