Tools · · 2 min read

llama.cpp 0.3.0 adds DeepSeek 4 tensor split — but the 50% gain is prompt-only

llama.cpp v0.3.0 adds -sm tensor for DeepSeek 4, fixes multi-sequence rollback, and adds a server slot-diff debug control. The sole published 4×4090 result reports 50% faster prompt processing, not faster generation.

ggml-org released llama.cpp v0.3.0 on August 25, 2026. The most consequential change for a multi-GPU DeepSeek 4 deployment is -sm tensor, a new tensor-split mode; the same release also fixes rollback with multiple active sequences and adds a server-side prompt-difference debug window. (Source: llama.cpp v0.3.0 release notes, 2026-08-25)

Key facts:

  • v0.3.0 was published on August 25, 2026.
  • DeepSeek 4 now has tensor-split mode through -sm tensor.
  • The PR author reported 50% higher prompt processing on four RTX 4090 GPUs.
  • The same author reported no token-generation increase in that test.
  • LLAMA_SERVER_SLOTS_N_DIFF prints a configurable token window around a slot-cache mismatch.
Official GitHub release notes for llama.cpp v0.3.0 listing DeepSeek 4 tensor split mode, multiple-sequence rollback fix, and GLM-4.5-Air MTP support.
The official v0.3.0 overview lists the DeepSeek 4 changes alongside the new model and multimodal work. Screenshot: ggml-org release notes.

What this means if you run DeepSeek 4 across several GPUs

Do not read this as a general “50% faster llama.cpp” claim. The number comes from the author of the tensor-split pull request, on one 4× RTX 4090 setup, and applies to prompt processing (prefill) only. The PR explicitly says token generation did not increase. (Source: PR #26490, merged 2026-08-24)

That distinction matters for coding agents. Long repository instructions, tool schemas, and retrieved files spend meaningful time in prefill; a prefill improvement can shorten time to first token. But an interactive agent that generates hundreds of tokens still needs you to measure generation speed, queue time, and memory use before changing a production topology.

The rollback fix is the less flashy update to prioritize if your server keeps several sequences in flight. The upstream bug was that a snapshot did not follow the semantics used by the recurrent snapshot path; v0.3.0 adds a DeepSeek 4 test case as part of the fix. (Source: PR #26756, merged 2026-08-23)

Official llama.cpp v0.3.0 release page showing ggml v0.22.0 changes, Metal per-operation source split with parallel compilation, and the version changelog.
v0.3.0 also brings ggml v0.22.0, including meta-backend split-state work and Metal compilation changes. Screenshot: ggml-org release notes.

One debugging knob worth knowing

For cache or slot reuse failures, start the server with both existing slot debugging and a narrow diff window:

LLAMA_SERVER_SLOTS_DEBUG=1 \
LLAMA_SERVER_SLOTS_N_DIFF=64 \
llama-server -m deepseek4.gguf

LLAMA_SERVER_SLOTS_N_DIFF=64 asks the debug log to show 64 tokens on either side of the first difference between old and new prompts. It is for diagnosis, not a performance flag. (Source: PR #27600, merged 2026-08-23)

If you are tuning several users rather than one model across GPUs, pair the upgrade with the llama-server concurrency guide and monitor slots instead of assuming a larger split improves every workload. If repeated prompts are the issue, the KV-cache reuse guide covers the server-side cache controls. Keep built-in file or shell tools in a constrained runtime; our safe --tools all guide shows a Docker boundary for that setup.

Sources

Source: ggml-org/llama.cpp v0.3.0 release notes