Gemini 4 Carbon tops AI benchmarks with guarded security rollout

By Rebecca Stone

Artificial Analysis scored Gemini 4 Carbon at 59 points on version 5.0 of its Intelligence Index, nine points ahead of Gemini 4 Argon’s 50, after Google DeepMind released the model two weeks after Argon.

Google has placed Carbon in the U.S. government’s voluntary pre-release testing process because of its cybersecurity capabilities. Selected security experts receive access first, rather than developers or consumers.

Artificial Analysis and Vals AI results also expose a tradeoff: Carbon improves accuracy and agentic work but consumes more tokens, runs some tasks for far longer, and raises hallucination rates against Argon.

What did Google change two weeks after Argon?

Google DeepMind says Carbon targets "much harder, long-horizon engineering problems" that demand persistent agentic workflows. Google says internal teams ran it for multiple days. The model worked through dead ends on its own and handled root-cause analysis, subtle bug fixes in large codebases and low-level software.

Google also reports improvements in scientific reasoning, multimodal understanding and full-stack web development, based on user feedback. Better outputs and artifacts in WebDev Arena lifted Carbon 89 Elo points to 1,769, placing it fourth.

On Gray Swan’s indirect prompt-injection benchmark, Google reported that Carbon reduced the attack success rate from 0.7% with Argon to 0.0%, blocking every tested manipulation. The Decoder said this does not mean prompt injection is solved; the test appears saturated.

Where does Carbon lead independent evaluations?

Artificial Analysis said Carbon at its highest reasoning tier, “high,” leads Intelligence Index v5 mainly through top scores in complex reasoning, knowledge retrieval and scientific tasks. The model placed first in four of the index’s nine component evaluations.

• Artificial Analysis’s Coding Agent Index places Carbon at 59 versus Argon’s 50; Claude Opus 5.5 remains first at 62.
• On Antigravity, Carbon climbed from 53 with Argon to 59, but Claude Code with Opus 5.5 stayed three points ahead.

In agentic knowledge work on AA-Briefcase, Carbon at “high” gained 126 Elo points over Argon at “high,” which had 1,483 Elo, reaching 1,609 Elo. Artificial Analysis attributed the gain to presentation quality, up 293 Elo to 1,598, and analytical quality, up 120 Elo to 1,671.

Carbon still ranked behind Kimi K3.1 Pro at 1,678 Elo, Muse Wonder at “max” with 1,659 Elo and Grok 4.7 at “xhigh” with 1,638 Elo.

Carbon’s answer accuracy rose from 48% to 58% against Argon, while its hallucination rate increased from 4% to 11%. Artificial Analysis still ranked Carbon’s hallucination rate second-lowest; Claude Opus 5.5 at “max” and GPT-6 Astra at “max” each recorded 50%.

Vals AI put Carbon second at 70.36% accuracy, behind Claude Sonnet 5.5 at 70.85% and Claude Opus 5.5 at 70.18%. Argon ranked fourth at 69.24%, and Vals AI said all three rivals cost less to run.

Carbon led Vals AI’s Finance Index and Legal Research benchmark. In legal research, Vals AI recorded roughly 45% more cited sources, more frequent tool calls and longer responses than Argon.

Vals AI also found that Finance and Tax Agent tasks took more than twice as long as with Argon, while Legal Research averaged more than 50 minutes per task. Carbon generated 2.3 times as many reasoning tokens and 64% more output tokens than Argon. That output remained less than half the count for Claude Sonnet 5.5 and Claude Opus 5.5.

Arena.ai placed Gemini 4 Carbon at “High” first in Text Arena with 1,528 points, eight ahead of Argon at 1,520. In Agent Arena, Carbon ranked second across more than 21,900 real sessions, ahead of GPT-6 Astra and Fable-5.1.

Carbon posted a 16.2% net improvement and the highest steerability score, up 18.6%. Its median task cost of $1.80 was 72% below first-place Claude Opus 5.5 at “Max.”

Separately, Arena.ai scored Carbon at 1,768 points in the WebDev category of Code Arena, 89 ahead of 11th-place Argon and two behind Claude Sonnet 5.5 at “xHigh” with 1,770. The result places Carbon on the Pareto curve across all three Code Arena disciplines, Arena.ai said.

What does the security testing show?

Google ran several security tests without safety guardrails. The company says Carbon covers the full cyber-defense lifecycle, from detecting and validating vulnerabilities to writing patches.

• On SEC-bench Pro, which measures autonomous vulnerability discovery in real-world software, Carbon scored 88.2% and led all tested models.
• On ExploitGym, which tests whether a system can turn discovered vulnerabilities into working exploits, Carbon reached 54.8%.
• On CWE-Bench, Carbon held a wide first-place lead in writing fixes.

Google also cited internal results that independent evaluators cannot verify. Its in-house benchmarks reportedly produced secure code more than twice as often as Argon. In tests by Google’s Chrome team, Carbon performed more than ten times better at writing security patches than the frontier model of an unnamed competitor.

Who gets Carbon first?

Google is rolling out Carbon gradually through the U.S. government’s voluntary pre-release testing process, citing the model’s advanced security capabilities. Selected cybersecurity experts receive access first starting this week.

Through Google DeepMind’s Fairwind Program, access will then expand to more trusted partners. Google plans a wider release through its product interfaces for developers, businesses and consumers after collecting more feedback and adjusting safety guardrails.

What does the model cost?

Carbon launches at the same introductory price as Argon: $2 per million input tokens and $10 per million output tokens. Cached input tokens receive a 95% discount from the regular input price.

Artificial Analysis calculated a cost of $7.76 per task, about 23% below Claude Opus 5.5 at “max,” which costs $10.04, and roughly 1.2 times the $6.25 cost of GPT-6 Astra at “max.” Carbon’s task cost was nearly identical to Argon’s $7.68.

Artificial Analysis measured 123,000 output tokens per Carbon task, 29% fewer than the Anthropic model’s 173,000. Those calculations use regular list prices of $4 per million input tokens and $20 per million output tokens, plus $0.20 for cached input. The platform ignores the current 50% introductory discount, which runs through the end of 2026.

Google’s next checkpoint is the broader release it plans after tester feedback and guardrail adjustments, with security performance and task cost setting the terms.