JetBrains' Mellum2.1 lifts SWE-bench score from 2.0 to 47.0 with pure RL

A single number tells the story of Mellum2.1: SWE-bench Verified jumped from 2.0 to 47.0. JetBrains achieved that leap without touching the architecture. The company released Mellum2.1, a 12B mixture-of-experts thinking model for coding agents, on October 8, 2026. It ships under Apache 2.0 on Hugging Face.

The upgrade came almost entirely from reinforcement learning in real software environments. Mellum2.1 activates only 2.5B of its 12B parameters per token. That makes it small enough to self-host, fast enough to run as a sub-agent, and cheap enough to keep on your own GPUs. The stakes are real: as coding agents eat more of the software workflow, teams want capable models they can run privately, without shipping source code to a third-party API.

What is Mellum2.1?

Mellum2.1 is the successor to Mellum2 Thinking, which JetBrains open-sourced in June 2026. The released checkpoint is Mellum2.1-12B-A2.5B-Thinking. It is a reasoning model that emits its chain of thought before answering. JetBrains targets three use cases:

• Agent worker for coding tasks
• General reasoning assistant
• Private self-hosted deployment

The model carries a 131,072-token context window and a 98,304-token vocabulary. Weights ship in bfloat16.

How does the architecture work?

The design is unchanged from Mellum2, and that is the point. The model has 28 layers and 64 experts. A router activates 8 experts per token. Attention uses grouped-query attention with 32 query heads and 4 KV heads. Three of every four layers use a 1,024-token sliding window. Because the architecture stayed frozen, speed carries over directly from the previous generation.

How was it trained?

Almost all of the new work went into post-training. RL moved from a short final stage to the main part of training. JetBrains added RL tasks in math, competitive programming, science, tool use, and software engineering. The team filtered open RL datasets for broken tests, unverifiable answers, and tasks that were too easy or impossible.

For software engineering, the training setup gets concrete. The model works in real repositories with a shell and file-editing tools. It gets rewarded when the tests pass. Training launched millions of sandboxes across thousands of environments.

How does it perform on benchmarks?

JetBrains evaluated Mellum2.1, Mellum2, Qwen3.5-9B and Gemma 4 E4B in one shared pipeline in thinking mode. All scores are self-reported by JetBrains.

The largest jump is agentic coding:

• SWE-bench Verified: 2.0 to 47.0
• SWE-bench Pro: 0.0 to 28.0
• Terminal-Bench 2.1: 0.6 to 17.4

Agentic runs used the open-source Pi v0.73.1 harness with a 114K-token context.

Mellum2.1 beats Mellum2 on 15 of 17 listed benchmarks, and wins 5 of 17 against Qwen3.5-9B. It leads the tested group on LiveCodeBench v6 (82.0), HumanEval+ (91.5), MBPP+ (79.4) and BFCL v4 (62.3). On LiveCodeBench v6, it sits ahead of Qwen3.5-9B (75.4) and Gemma 4 E4B (69.4).

The losses matter too. Qwen3.5-9B still leads on SWE-bench Verified (50.0), SWE-bench Pro (38.0), AIME 25/26 (86.7) and GPQA Diamond (77.8). Mellum2.1's weakest showing is Terminal-Bench 2.1 at 17.4, below Qwen3.5-9B's 21.7. In short: strong coding scores and high throughput from only 2.5B active parameters, but it still trails on hard agentic software tasks and knowledge.

Safety also improved. HarmBench fell from 21.5 to 8.5, where lower is better.

What about evaluation pipelines?

Pipelines are important to consider, and the gaps are not subtle. Qwen's own card lists 65.6 on LiveCodeBench v6 and 81.7 on GPQA Diamond. JetBrains measured 75.4 and 77.8 for the same model. That spread is a useful reminder that cross-model comparisons depend heavily on who runs them and how.

How fast is it?

Post-training left the architecture untouched, so speed matches Mellum2. On a single NVIDIA H200 under heavy load, JetBrains says Mellum2.1 serves almost 2x the tokens of Qwen3.5-9B. For a single request, multi-token prediction (MTP) makes it about 1.6x faster. The MTP head for vLLM speculative decoding is listed as coming soon.

How do you run it locally?

The full model serves on vLLM with --reasoning-parser qwen3. Tool calling adds --enable-auto-tool-choice --tool-call-parser hermes. JetBrains recommends temperature 0.6, top_p 0.95 and top_k 20.

The GGUF repository already lists five builds:

• BF16: 24.3 GB (reference)
• Q8_0: 12.9 GB, 96.1% top-token match
• Q6_K: 10.9 GB, 93.9% top-token match
• Q4_K_M: 8.1 GB, 88.0% top-token match (recommended)
• MXFP4_MOE: 7.0 GB, 85.6% top-token match

Those sizes work with llama.cpp, Ollama and LM Studio. A 7.0 GB to 8.1 GB file makes local coding sub-agents practical on consumer hardware.

Mellum2.1 vs Qwen3.5-9B vs Gemma 4 E4B

Feature	Mellum2.1	Mellum2 Thinking	Qwen3.5-9B	Gemma 4 E4B
Architecture	MoE, 64 experts, 8 active	MoE (same as 2.1)	Hybrid Gated DeltaNet + gated attention	Dense with per-layer embeddings
Total params	12B	12B	9B (language model)	8B with embeddings
Active params	2.5B	2.5B	9B (dense)	4.5B effective
Context	131,072	131,072	262,144 native, up to 1,010,000	128K
Modality	Text	Text	Text, image, video	Text, image, audio
License	Apache 2.0	Apache 2.0	Apache 2.0	Apache 2.0
LiveCodeBench v6*	82.0	69.4	75.4	69.4
SWE-bench Verified*	47.0	2.0	50.0	23.0
Terminal-Bench 2.1*	17.4	0.6	21.7	3.4
BFCL v4*	62.3	49.6	58.5	52.5
GPQA Diamond*	64.6	51.0	77.8	53.1
AIME 25/26*	83.3	60.1	86.7	45.0

*Benchmark rows come from JetBrains' shared pipeline, thinking mode, per the Mellum2.1 model card. Vendors' own cards report different numbers for Qwen3.5-9B and Gemma 4 E4B.

Why does this release matter?

The Mellum2.1 story is a data point in a broader shift. Post-training, not pretraining, drove a 23x improvement on SWE-bench Verified with zero architectural change. That rewards teams with strong RL infrastructure and access to real execution environments, not just massive compute budgets.

The model also signals where JetBrains is positioning itself. Mellum2.1 is not chasing frontier multimodal capability. Qwen3.5-9B handles text, image and video; Gemma 4 E4B handles text, image and audio. Mellum2.1 handles text, full stop. It is built to explore a repository, edit files, and check its own changes — the boring, high-frequency loop that coding agents run all day.

The throughput numbers strengthen that case. Serving nearly 2x the tokens of Qwen3.5-9B on one H200 means agent fleets get more work done per GPU-hour. And the Apache 2.0 license plus GGUF builds starting at 7.0 GB lower the barrier for private deployment.

The gaps define the roadmap. Mellum2.1 still trails Qwen3.5-9B on SWE-bench Pro, Terminal-Bench 2.1, GPQA Diamond and AIME. If JetBrains closes the agentic gap while keeping its throughput edge, the next release could pressure much larger models in the coding-agent niche they can't afford to ignore. — Elena Vasquez