Chips & Compute

Cerebras puts disaggregated AI inference into production

Cerebras says a production cloud setup using AMD Instinct GPUs for prompts and CS-3 for decoding delivers five times the throughput per megawatt, with rollout targeted by end of 2027.

Cerebras Puts Disaggregated Inference Into Production With GPUs
Cerebras Puts Disaggregated Inference Into Production With GPUsTho-Ge / pixabay
By James Calloway5 min read

Why it matters

  • On November 17, 2026, Cerebras reported 5× throughput per megawatt versus CS-3-only, without slowing token generation.
  • The November 14, 2026 test used a shared GPT-OSS-120B endpoint and a normalized 1× baseline.
  • Cerebras claims 350× the SRAM capacity and 5,400× the memory bandwidth of a typical GB300 GPU.
  • The partner package offers a full routing stack or decode-only SDK, with early access for Amazon Web Services, Fireworks AI, Gimlet Labs and General Compute.
  • Cerebras targets deployment across hundreds of megawatts by the end of 2027, not capacity already operating.

By James Calloway

Cerebras Systems reported on November 17, 2026, that its production disaggregated inference setup delivered five times the throughput per megawatt of a CS-3-only configuration “without reducing token-generation speed.” The deployment assigns prompt processing to AMD Instinct GPUs and response generation to CS-3 systems. The result applies specifically to Cerebras Inference Cloud.

Cerebras says the split frees CS-3 serving capacity for interactive token generation while serving more requests from the same power budget. That matters because the company targets power-constrained data centers, where extra capacity can support more customer traffic and revenue.

What changed from the early tests?

Cerebras measured the production configuration on November 14, 2026, using a shared GPT-OSS-120B endpoint. The company compared throughput per megawatt before and after enabling disaggregation and normalized the aggregated configuration to a 1× baseline.

The disclosure follows Cerebras’ October 1, 2026 explanation of disaggregated inference. That earlier account reported a 5× gain in early tests with the same wafer-scale-engine footprint. The November announcement adds a production measurement and a deployment package for infrastructure partners.

The available evidence remains narrow. Cerebras does not disclose absolute tokens per second, requests per second, latency, the number or exact model of AMD Instinct GPUs, or the request mix. It names no production customer and supplies no independent benchmark.

The announcement also omits request concurrency, energy-measurement boundaries and the exact endpoint configuration. Those omissions do not erase the reported ratio, but they make it difficult to judge performance outside the tested cloud configuration.

A production deployment can still represent one workload rather than a general result. Cerebras has moved beyond an early test, yet its disclosure stops short of a field-wide benchmark.

The result is useful production evidence for one cloud configuration, but it is not proof of universal fivefold efficiency.

Why separate prompt processing from token generation?

Cerebras divides each inference request into two principal stages. Prefill processes the input prompt and creates intermediate model state. Decode then uses that state to produce output tokens one at a time, feeding each new token back into the model.

Because every decode step rereads model data from memory, memory bandwidth—the rate at which a processor can read data—sits at the center of interactive generation speed. Cerebras says prefill instead performs more computation across the prompt in parallel, which makes it a better match for accelerators built for that load.

Its wafer-scale systems place model data in on-chip static random-access memory (SRAM) instead of relying primarily on off-chip high-bandwidth memory. Cerebras describes the chip as carrying “tens of gigabytes of SRAM and tens of petabytes per second of memory bandwidth.”

The company puts those figures at 350× the SRAM capacity and 5,400× the memory bandwidth of a typical GB300 GPU. The announcement gives no methodology for the “typical” configuration, so the comparison remains a Cerebras claim rather than an independently verified measurement.

Cerebras says the capacity lets more concurrent requests and model layers remain directly on the chip for rapid decode. Offloading prefill therefore reserves the wafer-scale system’s memory resources for the stage that repeatedly depends on them.

That arrangement only helps if the transferred workload adds little delay and the CS-3 systems keep generating tokens at the same speed. Cerebras says the production measurement met that condition.

The architecture reserves fast memory for token generation and assigns prompt computation to GPUs that fit that work better.

How does the deployment package work?

Cerebras provides a reference architecture for infrastructure partners. They can co-locate selected prefill accelerators and Cerebras systems on a unified network, then choose between two integration routes.

  • End-to-end software stack. This option routes requests between prefill and decode pools and manages replica capacity, meaning the capacity of serving copies, and health. Its prefill engine can run open-source inference engines such as vLLM or SGLang, while Cerebras supplies the decode engine.
  • Decode-only software development kit (SDK). Partners retain their own inference framework and prefill engine while using Cerebras’ decode engine and KV Portal.

The full stack is intended to shorten time to market. The decode-only route gives partners more control over the rest of their inference software. Neither option removes the need for the unified network between the hardware pools.

Cerebras’ KV Portal transfers intermediate model state over remote direct memory access (RDMA), which moves state directly between systems. That state includes key-value caches, which preserve request-specific context after prefill. The decode-only SDK also includes a mock, or test, decode replica and a reference integration with an open-source disaggregated inference framework and prefill engine.

Amazon Web Services, Fireworks AI, Gimlet Labs and General Compute are named as early-access partners. Cerebras says it is expanding access to additional partners. The announcement does not show partner production traffic or quantify integration overhead, so early access should not be read as proof of broad deployment.

Partners have two software routes into the architecture, but the disclosure stops short of demonstrating either one at scale.

What is the capacity target?

Cerebras plans to deploy disaggregated inference across hundreds of megawatts in its own inference cloud and partner deployments by the end of 2027. The company presents this as a deployment target, not capacity already operating.

The economic target is more customer-serving capacity and revenue potential from the same power budget. Cerebras cites Muse, Copilot and Codex as examples of rapidly scaling agentic applications that require highly interactive responses. For those workloads, added capacity helps only if response speed remains intact.

The central scaling question is narrower than the headline number. Cerebras measured one shared GPT-OSS-120B endpoint. It has yet to show whether the fivefold advantage survives across partners’ selected accelerators, frameworks and deployment sizes.

That uncertainty has two practical dimensions. Operationally, partners must move model state between pools, route requests and manage replica health without eroding the throughput gain. Commercially, Cerebras must fill the planned capacity with paying workloads and turn the measured power efficiency into revenue.

The announcement gives no customer pricing, production-user list or deployment schedule beyond the broad end-of-2027 target. It therefore supports a claim about one measured cloud setup and a software path for outside operators, but not a forecast of partner-scale economics.

Cerebras’ next test is whether the fivefold result survives across partner hardware and the hundreds of megawatts it plans to deploy by the end of 2027.

Source: unite.ai

Share this article:

More from James Calloway

James Calloway

Show full bio

News editor covering people, launches and day-to-day practice in artificial intelligence and machine learning at AI In Context.

722 articles

Related articles

  1. Cerebras Q3: Cloud Now 63% of Revenue, Microsoft Signs On
  2. Cerebras Raises Full-Year Outlook as Cloud Revenue Jumps 265%
  3. AI Compute in 2027: Serving Economics Will Reshape Data Centers
  4. Cerebras releases Tungsten for wafer-scale programming
  5. MemryX Targets 10x Compute Jump With MX3+ Edge AI Chip

« Previous articleNext article »