llama.cpp b11718 puts ESIMD kernels back on the SYCL FFN path: about 2% faster decode on B70, plus an open A770 fusion slowdown

llama.cpp b11718 (October 18, 2026) lets the SYCL backend's fused gate+up+GLU path use Intel's ESIMD DMMV kernels for Q2_K to Q6_K and Q8_0. The PR measured +0.9% to +2.7% tg128 on a B70. Separately, issue #30478 reports fused FFN running about 8% slower than unfused on an Arc A770. Here is what Intel GPU users should test.

llama.cpp b11718, published October 18, 2026, is a small SYCL release for Intel GPUs. Its only change is PR #30239, “Restore ESIMD kernels on FFN path”, from Intel’s Todd Malsbary. It is a decode-speed change. Prompt processing is not what it targets.

Builder question this answers: if I run llama.cpp on an Intel Arc GPU through SYCL, is b11718 worth pulling, and how do I check whether the newer fused FFN path helps or hurts on my card?

What changed

The SYCL backend recently gained an FFN fusion path. It computes the gate and up mat-vecs and the GLU activation in one fused step instead of separate kernels. According to the PR, b11718 “adds support to the recently added FFN fusion path for the existing ESIMD kernels”. ESIMD is Intel’s explicit-SIMD kernel style, and the backend already used it for unfused dequantize-mat-vec (DMMV) decoding.

From the merged code in ggml-sycl/fusion.cpp, the ESIMD fused path is taken only when all of these hold:

Gate and up weights are the same type, one of Q2_K, Q3_K, Q4_K, Q5_K, Q6_K or Q8_0. Q8_0 is the newly added type.
The GLU is SwiGLU or GEGLU.
There is a single activation column, meaning one decoded token at a time.
GGML_SYCL_ENABLE_ESIMD is on. It defaults to 1.
The build used Intel’s oneAPI compiler (icx/icpx). GGML_SYCL_DMMV_HAS_ESIMD is only defined under __INTEL_LLVM_COMPILER.

Other cases fall back to the existing reorder (same-type Q4_K) and plain-layout (Q5_K/IQ4_XS) fused kernels, or to the unfused ops.

The numbers in the PR

The author measured tg128 on a B70:

Model	Baseline	b11718	Change
gemma3 1B Q8_0	246.92	253.49	+2.7%
qwen2 3B Q4_K_M	169.45	173.81	+2.6%
gemma4 E2B Q8_0	128.24	131.35	+2.4%
llama 8B Q5_K_M	85.52	87.41	+2.2%
gemma2 2B Q4_K_M	182.88	186.91	+2.2%
qwen35 27B Q8_0	18.47	18.63	+0.9%

The PR also reports that llama-perplexity runs on gemma-2-2b-it Q4_K_M and qwen2.5-coder-3b-instruct Q4_K_M (SwiGLU and GEGLU) showed no significant change on hellaswag, wikitext and winogrande. These are one contributor’s results on one GPU. Expect a low single-digit gain at most, and less on large models.

The A770 caveat: issue #30478

While working on this PR, the same author opened issue #30478, which is still open. On an Arc A770 under Linux, gemma-2-2b-it Q4_K_M decoded at 81.26 tokens/s with FFN fusion and 87.94 tokens/s with it manually disabled. That makes the unfused path about 1.08x faster. A Q2_K version of the same model that fits in L2 cache showed no difference (102.18 vs 102.03). The issue does not reproduce on the B70.

The author’s explanation is that when the weights touched by the fused kernel approach the L2 size, the following flash-attention kernel slows down. An experimental patch that split fused launches to about 3/4 of L2 brought decode up to 90.28 tokens/s, but it is not merged. With this PR applied, the fused number was 82.67, so b11718 does not fix the A770 case. A maintainer suggested an environment variable to control FFN fusion. That is still under discussion.

How to test on your card

Build per the SYCL backend docs, or grab the SYCL archives on the release page:

source /opt/intel/oneapi/setvars.sh
git fetch --tags && git checkout b11718
cmake -B build -DGGML_SYCL=ON -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icpx -DGGML_SYCL_F16=ON
cmake --build build --config Release -j
./build/bin/llama-bench -m your-model-Q4_K_M.gguf

Then A/B with the documented runtime switches. GGML_SYCL_ENABLE_ESIMD=0 turns ESIMD kernels off. GGML_SYCL_ENABLE_FUSION=0 turns off fused-kernel dispatch. Note that this flag disables every SYCL fusion, not just the FFN one, so it is a blunter switch than the manual change in the issue:

GGML_SYCL_ENABLE_FUSION=0 ./build/bin/llama-bench -m your-model-Q4_K_M.gguf

If tg128 is higher with fusion off on your A-series card, keep that setting for serving until #30478 lands. After that, check end-to-end numbers with our llama-server streaming latency guide. For Intel’s other llama.cpp path, see our write-up on b10870’s Vulkan matmul tuning for Intel GPUs. For a recent decode-kernel change on NVIDIA, see b11713’s CUDA MMVQ Q2_K/Q3_K work.

Sources: llama.cpp b11718 release · PR #30239 · issue #30478 · SYCL backend docs · ggml-sycl fusion.cpp at b11718