Agentic AI Workloads Could Quadruple Data Center CPU Core Demand

Arm estimates cited by TrendForce put the shift in concrete terms: CPU core requirements could rise from approximately 30 million cores per gigawatt of data center capacity to 120 million cores per gigawatt as agentic AI workloads spread. That is a fourfold increase, and it comes for the processor that the AI industry spent the last three years treating as an afterthought.

Semiconductor research firm TrendForce laid out the finding in a recent analysis of how autonomous AI agents are changing server infrastructure. The core claim is straightforward. AI agents introduce a more CPU-intensive workload than conventional large language model inference. GPUs remain essential for training AI models, but CPUs coordinate the complex sequences of actions that let agents operate autonomously — steps, TrendForce notes, that a GPU simply cannot do.

That coordination work is not marginal. TrendForce breaks it down: planning tasks, scheduling subtasks, calling external tools, managing interactions between multiple agents, and evaluating results. Agents also lean on conventional computing resources to execute Python code, search databases, crawl websites, and process information. Each of those operations runs on general-purpose silicon, not on accelerators.

The result, according to TrendForce, is that the CPU is enjoying a new lease on life. It is becoming a significant component of AI infrastructure in its own right rather than a supporting processor for GPUs.

How does the CPU-to-GPU ratio change?

The numbers define the story. Traditional AI data centers have deployed approximately one CPU for every four to eight GPUs, a ratio that reflects the industry's emphasis on accelerating computationally intensive AI workloads — training above all.

Agentic AI could change that. TrendForce expects the CPU-to-GPU ratio in agentic AI infrastructure to move toward one CPU for every one to two GPUs. That is a shift from a sparse complement of host processors to near-parity deployment, and it follows directly from the Arm projection of 120 million cores per gigawatt of capacity.

The ratio matters because it shapes what operators buy, how they design racks, and how they budget. A move from 1:8 toward 1:2 means CPUs stop being a rounding error in data center procurement and become a primary line item.

Where is the bottleneck?

The performance case for rebalancing rests on latency data. Research cited by TrendForce indicates that CPU-based tool processing can account for as much as 90.6% of total latency in some agentic workloads. Nine-tenths of the wait, in other words, sits on the processor coordinating the agent, not on the accelerator running the model.

Energy follows a similar pattern. CPU dynamic energy consumption can represent a substantial portion of total system energy use under certain operating conditions, TrendForce reports.

The implication is direct. Increasing GPU capacity alone may not improve overall performance if the processors responsible for coordinating AI agents and executing their associated tasks become bottlenecks. Operators who bought accelerators on the assumption that everything else would keep up now face a different arithmetic: system-level throughput, energy draw, and infrastructure cost all hinge on the CPU side of the house.

What is Nvidia doing about it?

The clearest signal of the CPU's resurgence comes from Nvidia. For years, the company downplayed and even disparaged CPUs, positioning the GPU as the be-all and end-all of AI computing. That position has eroded.

Nvidia changed course a few years ago with the introduction of the Grace processor, now on the Vera generation. Vera features 88 CPU cores and 176 threads, and it pairs with Nvidia's GPU platforms through a high-bandwidth NVLink-C2C interconnect that handles communication and memory sharing with the company's accelerators.

The company is also developing dedicated Vera CPU racks. Those let customers deploy CPU systems independent of GPU-heavy or GPU-only systems. The move says plainly what Nvidia's marketing once denied: the CPU is needed to do things the GPU cannot.

A vendor that built its dominance on accelerators now sells general-purpose processors in their own racks. That is the market reading its own tea leaves.

Why does this matter beyond the data center?

TrendForce's analysis points to a broader change in AI computing. Systems are moving from generating individual responses toward executing multistep tasks autonomously. As they do, the processors coordinating those tasks become increasingly important.

This reframes a debate that had seemed settled. The GPU shortage era trained the industry to measure AI capacity in accelerators. If agents become the dominant enterprise workload — planning, calling tools, running code, coordinating with other agents — then CPU supply, core counts, and interconnect bandwidth become constraints of equal practical weight.

The stakes run in three directions. For chipmakers, a fourfold increase in cores per gigawatt is a demand signal measured in tens of millions of units. For data center operators, the 90.6% latency figure argues that unbalanced infrastructure wastes accelerator investment. For the broader market, Nvidia's Vera CPU racks show that the biggest beneficiary of the GPU era now hedges toward general-purpose silicon.

None of this displaces the GPU. Training still runs on accelerators, and TrendForce says so explicitly. What changes is the ratio, the budget, and the definition of what counts as AI infrastructure.

The direction of travel is legible in one comparison: a data center that once needed 30 million CPU cores per gigawatt now plans around 120 million. Whether operators rebalance early or discover the bottleneck in production, the agents themselves will keep issuing the workload that forces the decision.