JetBrains Mellum2.1 Thinking locally: 7–13 GB GGUF builds, llama.cpp and vLLM commands, and where the 12B MoE falls short

JetBrains’ Mellum2.1 Thinking is an open-weight coding and reasoning model under the Apache 2.0 license. JetBrains published official GGUF builds on October 7, 2026, so it now runs in llama.cpp, Ollama and LM Studio as well as vLLM. Developers on r/LocalLLaMA are now asking whether it’s worth trying. The short answer: it’s small enough for an 8–16 GB GPU, but JetBrains’ own numbers show it behind Qwen3.5 9B on the hardest agentic benchmarks.

Key facts:

• Architecture. 12B total parameters, 2.5B active, 64 experts with 8 active per token. It has 28 layers, GQA with 32 query and 4 KV heads, and sliding-window attention (1,024 tokens) on 3 of every 4 layers. Context is 131,072 tokens. (Source: model card)
• What changed from Mellum2. The architecture is the same as Mellum2. JetBrains says almost all the new work went into post-training, mainly reinforcement learning. That includes software-engineering tasks run inside real repositories with shell and file-editing tools, rewarded when tests pass.
• Output format. It’s a thinking model. It writes its reasoning inside <think>...</think> before the final answer.

Which file to download

All sizes and quality figures below come from JetBrains’ GGUF card. KLD is measured against BF16 on Wikitext-2 at n_ctx=512, so lower is closer to the original.

Quant	Size	KLD vs BF16	Top-token match
BF16	24.3 GB	—	—
Q8_0	12.9 GB	0.008	96.1%
Q6_K	10.9 GB	0.020	93.9%
Q4_K_M (recommended)	8.1 GB	0.075	88.0%
MXFP4_MOE	7.0 GB	0.115	85.6%

These are file sizes only. JetBrains doesn’t publish VRAM figures, and the full 131,072-token context adds KV cache on top of the weights.

Run it

llama.cpp, using JetBrains’ recommended sampling:

llama-server -hf JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF:Q4_K_M --ctx-size 131072 --temp 0.6 --top-p 0.95 --top-k 20

That serves an OpenAI-compatible API at http://localhost:8080/v1. With Ollama:

ollama run hf.co/JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF:Q4_K_M

With vLLM and tool calling (drop the last two flags if you don’t need tools):

vllm serve JetBrains/Mellum2.1-12B-A2.5B-Thinking --max-model-len 131072 --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser hermes

If you’re on vLLM 0.32, check our vLLM 0.32.0 upgrade notes first. That release made max_tokens a cap rather than a reservation against the context window, which matters here because JetBrains’ quickstart sets max_tokens=81920 so reasoning has room to finish. It also renamed the Responses API reasoning events. The model card also says an MTP head for speculative decoding in vLLM is “coming soon.”

How it compares

JetBrains ran every model through the same pipeline in thinking mode, and all figures are self-reported. The agentic runs used the open-source Pi v0.73.1 harness with a 114K-token context.

Benchmark	Mellum2.1	Mellum2	Gemma 4 E4B	Qwen3.5 9B
LiveCodeBench v6	82.0	69.4	69.4	75.4
SWE-bench Verified	47.0	2.0	23.0	50.0
SWE-bench Pro	28.0	0.0	4.0	38.0
Terminal-Bench 2.1	17.4	0.6	3.4	21.7
BFCL v4	62.3	49.6	52.5	58.5
GPQA Diamond	64.6	51.0	53.1	77.8

The jump from Mellum2 on agentic tasks is large, from 2.0 to 47.0 on SWE-bench Verified. It leads this group on LiveCodeBench and BFCL. Qwen3.5 9B still wins on SWE-bench Pro, Terminal-Bench and GPQA Diamond. If you need a stronger local coding agent and have a 24 GB card, a bigger MoE like Qwen3.6-35B-A3B is the step up. Mellum2.1’s case is size: the recommended Q4_K_M file is 8.1 GB.