llama.cpp b11965 fixes DeepSeek V4 fixed-batch graph reuse—not general inference speed
llama.cpp b11965 removes a DeepSeek V4 simple-batch path that could rebuild the compressed-KV graph on fixed-shape requests. The upstream report measured an 8.4% gain only for a specific three-token decode on eight RTX A6000 GPUs.
ggml-org published the pre-release llama.cpp b11965 after merging PR #31300 on October 28, 2026. It removes a DeepSeek V4 (DSV4) simple-batch splitting path that could make a fixed-size batch rebuild its compressed-KV graph. (Source: llama.cpp b11965 release notes, 2026-10-28)
Key facts:
- The affected architectures are DeepSeek V4 and DeepSeek V4.1.
- The regression test covers 3-, 5-, 6-, and 7-token sequences.
- The test covers both one and two sequences in a fixed 64-token micro-batch.
- The upstream report measured an 8.4% improvement only for a fixed three-token decode on eight RTX A6000 GPUs.
- A four-token decode was unchanged in that same report.
What b11965 changes
Before this patch, the split_simple() path recorded one token per sequence even when a sequence contained several tokens. For DeepSeek V4’s compressed cache, that mismatch could alternate compressor block counts inside a nominally fixed batch and force llama.cpp to rebuild the compute graph. b11965 removes that path and uses the existing sequence-aware splitter instead. (Source: PR #31300, merged 2026-10-28)
This is a narrow but useful distinction for builders: it concerns graph reuse within a fixed batch shape, not model quality, memory capacity, or a universal tokens-per-second upgrade. The added comp_padding regression case explicitly checks that later decode steps increment the graph-reuse counter rather than rebuilding the compressed-cache graph. (Source: PR #31300 changed files, 2026-10-28)
Who should update
Update first if you serve DeepSeek V4 or V4.1 through a recent llama.cpp build and send stable, short, multi-sequence decode batches—for example, a scheduler that repeatedly packs equal-length agent turns. The upstream validation says the new case failed before the change and passed after it on both CPU and CUDA. It also reports byte-identical logits in its comparison set, so the intended result is stable reuse rather than different output.
Do not infer an 8.4% gain for an ordinary single-user chat. That number came from one full Q8_0 DeepSeek V4.1 run using eight RTX A6000 GPUs, a fixed three-token decode, layer splitting, and 512/8,192-token prompts. A four-token decode did not improve. Treat it as evidence that avoiding needless rebuilds can matter under that configuration—not as a benchmark for Macs, consumer GPUs, or variable-length traffic.
For a production upgrade, pin b11965 or a later build in staging, replay your real mixed-length requests, and compare time to first token, decode rate, and graph-reuse telemetry separately. If repeated prompts remain slow, check cache policy and request shape with our llama-server KV-cache reuse guide. For shared endpoints, pair that check with the llama-server multi-user guide before changing slots or context limits.