Saluki 27B: A 7.89 GB, 2-Bit Qwen3.8 That Beats the 54 GB Original at Tool Calling

Underdog has squeezed Qwen3.8-27B into a 7.89 GB file that beats the original 54 GB model at tool calling. The release, called Underdog Saluki 27B, is a 2-bit, mixed-precision GGUF published under Apache 2.0 by Conway Research, the company behind the Underdog on-device assistant. On Underdog's 120-task tool-calling benchmark built from BFCL v4, Saluki scores 88 while the full BF16 model scores 84.

The numbers matter because tool calling is the skill that turns a chat model into an agent. Developers get a 27B-class agent model that runs in stock llama.cpp, with full GPU offload, on hardware that could never hold the 54 GB original.

What exactly is Saluki 27B?

Saluki stacks three layers of work on top of each other:

• The base: Qwen3.8-27B, a dense model from the Qwen team. It has 64 layers, mixes Gated DeltaNet linear attention with gated attention, and supports 262,144 tokens natively.
• The quantization layer: ISTA-DASLab's Qwen3.8-27B-GSQ-RCO-GGUF. GSQ learns accurate low-bit scalar grids per tensor; RCO assigns a quantization type to each tensor under a fixed size budget. ISTA's smallest file, IQ2_XS, is 8.4 GB at 2.50 bits per weight.
• Underdog's own pass: the company shrank the file further to 7.89 GB and tuned the compression to protect tool calling. The file is named IQ2-mix and carries an imatrix tag. Underdog has not published the full recipe.

That final, undisclosed pass is what separates Saluki from a standard quantization release — and it's where the tool-calling gains come from.

How does it perform?

Underdog reports 96% average retention across 9 benchmarks versus full Qwen3.8-27B. The spread, though, is wide. The company split its results into two groups.

In the first group, both models ran in the same harness:

• Underdog Bench (120 tasks from BFCL v4, frozen before testing, thinking off, temperature 0): Saluki 88, full model 84, PrismML's Bonsai 2 scores 70.
• Parallel tool calls (100 BFCL v4 parallel tasks, official checker): Saluki 42, full model 35 — 120% retention.
• SWE-bench Verified (50 issues): Saluki fixes 30, the full model 33.

The second group compares Saluki against public full-size scores:

Benchmark	Saluki 27B	Qwen3.8-27B (public)
IFEval (prompt-loose)	93.5	91.5
IFBench (prompt-loose)	72.7	71.0
MBPP+	78.0	83.9
MuSR	67.5	79.6
AIME 2025 (avg@4)	79.2	96.7
AIME 2026 (avg@4)	80.0	94.6

The pattern is clear: instruction following holds up or improves, parallel tool calls improve, and competition math plus multi-step reasoning drop 12 to 18 points. AIME 2025 falls from 96.7 to 79.2, about 82% retention — the worst result in the set. If your workload is agentic tool use rather than math olympiads, the trade looks favorable.

How does it compare with other compact Qwen3.8-27B builds?

Feature	Saluki 27B	Qwen3.8-27B (BF16)	ISTA GSQ-RCO IQ2_XS	PrismML Bonsai 2 (PTQ1_0)
Org	Underdog (Conway Research)	Qwen team	ISTA-DASLab	PrismML
Parameters	27B	27B	27B	27.36B
File size	7.89 GB	54 GB	8.4 GB	5.95 GB
Bits per weight	Not disclosed (2-bit mix)	16	2.50	1.75
Context	Not disclosed	262,144 native	Not disclosed	262K
Vision	Add-on, 629 or 928 MB	Native	0.9 GB mmproj	Optional 0.63 GB
Runtime	Stock llama.cpp	Transformers, vLLM, SGLang	Stock llama.cpp, Ollama, LM Studio	PrismML llama.cpp fork
Underdog Bench (of 120)	88	84	Not disclosed	70
License	Apache 2.0	Apache 2.0	Apache 2.0	Apache 2.0

Bonsai 2 is smaller and reports 98.2% retention across 14 thinking-mode benchmarks, with stronger math — 95.00 on AIME25. But it requires PrismML's llama.cpp fork, since stock llama.cpp rejects its packing formats. Saluki runs anywhere stock llama.cpp runs. One caveat: each vendor uses its own harness, so cross-vendor scores aren't directly comparable.

Why this matters

The local-agent market is the stakes here. A 27B model at 7.89 GB fits on consumer hardware — laptops, single-GPU boxes — and still handles the tool-calling loop that agent frameworks depend on. Saluki's pitch is that compression doesn't have to be uniform: you can spend your precision budget where the workload lives, and accept losses in math you'll never run.

Underdog hasn't disclosed the exact bits-per-weight or context window for the file, and the tuning recipe is unpublished. For now, the model card on Hugging Face, the Underdog launch page, and the announcement on X carry everything the company has shared. If the undisclosed recipe holds up under independent testing, expect targeted quantization to become a standard move for anyone shipping local agent models.