llama.cpp b11709 adds Aleph Alpha's Kolibri-1: how to convert the 78B-A3B model to GGUF and what it takes to run

llama.cpp build b11709 merged native support for Kolibri-1, Aleph Alpha's Apache 2.0 German/English MoE reasoning model (78B total, 3.46B active). The upstream author converted it to a 45 GiB Q4_K_M and measured 106 tokens/s generation on 4x A800. Here is the conversion path and the caveats.

On October 17, 2026, ggml-org merged PR #30673, “model : add Kolibri-1 (Aleph-Alpha) support”, and shipped it as llama.cpp build b11709. Aleph Alpha’s official serving path for its open-weight model is vLLM with the company’s own plugin. With b11709 or later you can convert the Hugging Face checkpoint to GGUF and run it with llama-server. (Source: llama.cpp PR #30673, merged 2026-10-17)

Key facts:

Model: Kolibri-1, released October 3, 2026 under Apache 2.0. It is a German/English mixture-of-experts reasoning model with tool calling.
Size: 78B total parameters, 3.46B active per token, 50 layers, 384 routed experts plus one shared expert.
Upstream test: the BF16 GGUF is 156 GiB and Q4_K_M is 45 GiB. Q4_K_M ran at 2558.18 tokens/s for pp2048 and 106.01 tokens/s for tg512 on a 4x A800 server.
Context: Aleph Alpha validated up to 1,048,576 tokens and recommends 262,144 or fewer for serving.

What the llama.cpp patch actually adds

The PR registers a new kolibri1 architecture. The converter reuses llama.cpp’s existing AFMoE code for the expert layout and adds the shared expert and the per-layer sliding-window pattern. Attention interleaves four sliding-window layers (window 513) with one full-attention layer. The sliding layers use NeoX RoPE and the full layers use no positional encoding, with 48 query heads against 4 KV heads.

Two details matter for anyone converting. First, the official checkpoint is FP8, and the PR says it converts through the generic weight_scale_inv path with no special handling. Second, the 128k vocabulary is mapped to the existing qwen2 pre-tokenizer by hash.

The PR touches only the converter, the gguf-py constants, the model code and tests. It adds no Kolibri-specific reasoning or tool-call parser, unlike vLLM, where the model card serves with --reasoning-parser kolibri1 --tool-call-parser kolibri1. In llama.cpp the chat template embedded in the GGUF does that work, so test tool calls before you build an agent on them.

Converting and running it

Update to b11709 or newer, download the checkpoint (about 78 GB of FP8 weights according to the model card), then convert and quantize:

hf download Aleph-Alpha/Kolibri-1 --local-dir Kolibri-1
python convert_hf_to_gguf.py Kolibri-1 --outtype bf16 --outfile kolibri-1-bf16.gguf
./build/bin/llama-quantize kolibri-1-bf16.gguf kolibri-1-Q4_K_M.gguf Q4_K_M
./build/bin/llama-server -m kolibri-1-Q4_K_M.gguf --jinja \
  --temp 1.0 --top-p 0.97 --top-k 128

The sampling values are Aleph Alpha’s recommended temperature=1.0, top_p=0.97 and top_k=128. The model card controls thinking through chat-template arguments: reasoning_effort set to low, medium, high or none, or enable_thinking=false. In llama-server those go through --chat-template-kwargs, for example '{"reasoning_effort":"medium"}'.

Plan disk space for the 156 GiB BF16 intermediate as well as the source weights. Because only 3.46B parameters are active per token, the expert weights are a good fit for --n-cpu-moe when the 45 GiB Q4_K_M will not fit entirely in VRAM. That is the same trade-off we covered for Nemotron 3.5 Bolt on 24GB cards. Aleph Alpha’s own FP8 vLLM recipe needs at least 2x A100 80 GB, 2x H100 or one H200, B200 or B300.

Community GGUFs predate the merge

Hugging Face already lists Kolibri-1 GGUF repos uploaded from October 3 onward, including Hob-forge/Kolibri-1-GGUF. Those were made before mainline llama.cpp had a kolibri1 architecture. If one fails to load on b11709, convert the official checkpoint yourself rather than debugging someone else’s fork.

Kolibri-1 is a sliding-window hybrid, so long multi-turn chats in llama-server depend on context checkpoints to avoid re-reading the whole prompt. Our context checkpoints guide explains how to tune them. For another large open MoE released this week, see what it takes to run Muse Spark Open locally.

Sources: llama.cpp PR #30673 · llama.cpp b11709 release · Kolibri-1 model card · Aleph Alpha: Kolibri has landed · llama.cpp feature request #29922