Alibaba's QwenGyre Cuts Long-Horizon Agent Training Time by Up to 1.85x

Alibaba's Qwen team says its new reinforcement learning framework, QwenGyre, trains long-horizon coding agents up to 1.85 times faster than Colocate setups while matching final scores. The framework, described in a paper on arXiv, targets what the researchers call "XLong-horizon" tasks: agent runs that stretch for hours and process close to one million tokens. Qwen tested it on its flagship Qwen 3.8 2.4T model, and the largest run finished 48 training steps in 75.42 hours, against 91.47 hours for Colocate and 134.55 hours for Async.

The stakes are practical. Training agents that operate real harnesses like Claude Code is expensive precisely because rollout times vary wildly and harnesses branch into sub-agents, which multiplies compute cost. QwenGyre attacks both problems, and it does so through unmodified, black-box harnesses — a design choice that matters for anyone training agents on tools they cannot instrument.

There is a catch, and it is a familiar one: the paper links no public code release, and the license is not disclosed. Competing frameworks veRL, AReaL and Agent Lightning are all open source under Apache-2.0 or MIT licenses.

What problem does QwenGyre solve?

The Qwen team defines XLong-horizon tasks as agent executions spanning hours and hundreds of model and tool interactions. Generating an entire repository from a natural-language specification, as in NL2Repo-Bench, is the canonical example.

Two problems appear at that scale. First, rollouts have long tails. In a Colocate setup (where training and inference share the same GPUs), training waits for the slowest execution while GPUs idle. In an Async setup, fixed GPU pools cannot lend spare capacity to each other.

Second, harnesses like Claude Code compact context, spawn sub-agents and retry. One execution becomes a branching tree rather than a single trajectory. Expanding every path duplicates shared prefixes and inflates training cost — a cost that grows with model size.

How does the elastic scheduler work?

QwenGyre splits GPUs into cells that share a training layout. Each cell can serve rollouts or host a full training replica. A core cell owns the authoritative weights and optimizer state; satellite cells join training later.

The scheduler tracks a "rollout waterlevel" — the count of queued and running executions. As executions finish, cells switch to training once the remaining cells can still serve all live work. Model calls are rerouted through a proxy, so the harness keeps running without noticing the switch.

The mechanics behind the switching are concrete:

• KV-cache blocks (the stored attention context that lets a model resume generation cheaply) migrate over RDMA during rerouting, reducing prefix recomputation.
• Satellite cells pull core weights through Mooncake's Transfer Engine and join an ongoing batch.
• Streaming training lets cells claim micro-steps as data becomes ready.
• Role switches averaged 8.52 seconds from rollout to training and 3.46 seconds the other way.

In plain words: the framework reassigns idle GPUs to whichever phase needs them, and the agent never stops mid-task.

How does the trajectory processor build training samples?

A proxy records exact token IDs and behavior log-probabilities using token-in, token-out (TITO), an off-policy correction technique. Calls are stored as prefix-shared trajectory trees rather than flat sequences.

Four rules keep the resulting samples bounded:

• Partial scoring: Timed-out executions are still scored on their preserved workspace. Evaluator failures are excluded entirely, never treated as a zero reward.
• Bounded sampling: At most 5 trajectories per execution are kept, ranked main > main-summary > sub-agent > sub-agent-summary.
• Deduplication: Shared targets count once, and only policy-generated tokens are trained.
• Fair weighting: Losses average within each execution, so extra branches do not inflate an execution's weight.

The cap matters. A cap of 5 matched uncapped training scores while cutting forward-backward time by 25.2%, according to the paper. Training uses an adapted GSPO (Group Sequence Policy Optimization) with 16 rollouts per group.

Takeaway: branching histories become a bounded, deduplicated set of samples rather than a combinatorial blowup.

How fast is QwenGyre in practice?

All benchmark runs used Claude Code 2.1.220 as the harness, with equal GPU budgets and matched staleness (the delay between when a rollout is generated and when it trains). Megatron handled training; SGLang served rollouts.

Workload (model)	vs Async	vs Colocate
NL2RepoBench (Qwen 3.6 122B, 48 steps)	1.42x–1.53x	1.36x–1.47x
DeepSWE (Qwen 3.6 122B, 24 steps)	1.569x–1.572x	1.610x–1.824x
TerminalBench (Qwen 3.6 122B, 24 steps)	1.383x–1.430x	1.809x–1.849x
NL2RepoBench (Qwen 3.8 2.4T, 48 steps)	1.78x	1.21x

The headline number is the TerminalBench result: up to 1.85x end-to-end speedup over Colocate. The weakest result is also worth staring at. On the 2.4T model, the advantage over Colocate shrank to 1.21x, and the paper attributes this to many queries running near the timeout — meaning less idle capacity existed to reclaim in the first place.

Quality did not pay for the speed. On the 2.4T run, evaluation pass rate rose from 52.48% to 58.54% over 48 steps on NL2RepoBench.

The compute footprint is substantial. The 122B runs used 24 to 32 nodes; the 2.4T run used 192 nodes arranged as 4 cells of 48. The GPU model is not disclosed. Rollouts averaged roughly 700K tokens per execution.

How does QwenGyre compare with other agentic RL frameworks?

QwenGyre enters a field with established open-source options. The paper's own comparison table covers veRL (ByteDance Seed plus community), AReaL (Tsinghua IIIS and Ant Group) and Agent Lightning (Microsoft).

• License: QwenGyre not disclosed; veRL and AReaL are Apache-2.0; Agent Lightning is MIT.
• GPU placement: QwenGyre uses elastic cells that switch mid-rollout; veRL uses flexible device mapping with HybridEngine resharding; AReaL is fully asynchronous; Agent Lightning does not disclose.
• Black-box harness support: QwenGyre via proxy; AReaL via base_url swap; Agent Lightning with near-zero code change; veRL does not disclose.
• Training/rollout stack: QwenGyre runs Megatron for training and SGLang for rollouts; veRL and AReaL support broader combinations.
• Largest stated scale: QwenGyre at Qwen 3.8 2.4T; veRL at 671B models; others undisclosed.
• Headline speedup: QwenGyre claims up to 1.85x vs Colocate; AReaL claims 2.77x vs synchronous in v0.3.
• Public code: QwenGyre, no; the other three, yes.

The paper's authors note, correctly, that these speedups come from each project's own setup and are not directly comparable. That caveat deserves emphasis: cross-framework speedup claims are effectively unverifiable without shared benchmarks.

The scale claim is the strongest differentiator on paper. QwenGyre's 2.4T-parameter run exceeds veRL's stated 671B by a wide margin — but without released code or an outside replication, that claim rests on Alibaba's own reporting.

What is missing from the announcement?

Three gaps stand out. No public code. No disclosed license. No disclosed GPU model. For a systems paper whose central contribution is infrastructure — an elastic scheduler and a trajectory processor — the absence of an artifact makes independent verification impossible.

The benchmark picture also has soft spots. All results come from Qwen's own runs at matched staleness, using one harness version (Claude Code 2.1.220). The 1.21x result on the largest model suggests the speedup degrades exactly where rollout tails shorten, which is worth watching as tasks and timeouts vary.

For now, QwenGyre is best read as evidence that elastic resource scheduling and bounded trajectory sampling can cut long-horizon RL training costs substantially at frontier scale — and as a signal that Alibaba is training 2.4T-parameter agents on hour-long, million-token tasks. Whether the techniques transfer beyond Qwen's clusters depends on a code release the paper does not promise.