How to Run Muse Glimmer 30B Locally: Ollama, llama.cpp, and vLLM

Meta released Muse Glimmer on August 10, 2026 under Apache 2.0. It is a 30B dense multimodal model built specifically for agent loops that run on your own hardware — the kind that stay resident all day, call tools, and never send a token to an API.

This guide gets it running four different ways, then tells you honestly where it beats the alternatives and where it does not.

Key facts:

  • The license is Apache 2.0, covering all released artifacts. No revenue ceiling, no territorial restriction.
  • The model is ~29.6B parameters, dense, including a ~1.8B ViT-G/14 vision encoder.
  • Context window is 131,072 tokens, extensible to 262,144.
  • The smallest usable quant is 15.9 GB (UD-Q4_K_XL); the smallest published is 10.7 GB (UD-IQ2_XXS).
  • Recommended sampling is temperature 1.0, top_p 0.95, top_k 64.
  • Reasoning effort is controllable across four levels: low, medium, high, xhigh.
  • A built-in DFlash drafter gives 1.5x–3.1x speedup depending on hardware.

(Sources: Meta Superintelligence Labs announcement, meta-models/Muse-Glimmer-30B model card, Unsloth docs, all retrieved 2026-08-11.)

How this guide was made: every command below is quoted from the official Meta / Unsloth / Ollama documentation, not re-run on our hardware — a 30B model needs more memory than our test machine has. Commands are copy-paste faithful to the source, but verify tags and filenames against the live docs before a long download, since these move fast.

1. Decide which quant you need before you download anything

Unsloth publishes 14 quantization levels. Most guides tell you to grab Q4 and move on. That is usually right, but the spread matters here because the vision tower does not degrade the same way the language layers do.

QuantFile sizeRealistic target
UD-IQ2_XXS10.7 GB12 GB VRAM, text-only, expect real quality loss
UD-IQ2_M12.3 GB16 GB VRAM, tolerable for simple tool routing
UD-Q3_K_XL13.4 GB16 GB VRAM, the honest floor for agent work
UD-Q4_K_XL15.9 GBThe default. 24 GB VRAM or 32 GB unified memory
UD-Q5_K_M19.2 GB24 GB VRAM, small quality gain over Q4
UD-Q6_K_XL26.3 GB32 GB VRAM
Q8_029.6 GB32–48 GB, near-lossless
BF1655.7 GB64 GB+, reference quality

(Source: unsloth/Muse-Glimmer-30B-GGUF, retrieved 2026-08-11.)

Meta’s own guidance is framed in VRAM envelopes rather than file sizes: the K-Quant-17GB build targets 24 GB VRAM at 1.0% measured degradation, and K-Quant-Dynamic targets 32 GB at 0.2%. Full BF16 needs 55+ GB. (Source: Muse Glimmer-30B model card, retrieved 2026-08-11.)

Rule of thumb: budget your file size plus roughly 4–6 GB for KV cache at moderate context. At 128K context the KV cache is not small, and the model’s [Local, Local, Local, Global] attention pattern with a 2,048-token sliding window means three of every four layers are cheap — but the fourth is not.

If you want vision, you need a second file. The GGUF repo ships mmproj-BF16.gguf separately. Without it, image inputs silently do nothing. This trips up nearly everyone on their first run.

2. The fastest path: Ollama

If you want it working in one command and do not care about tuning:

ollama run muse-glimmer

That pulls the 18 GB default build with a 128K context. On Apple Silicon, use the MLX variant instead — it is a larger download but runs materially better on Metal:

ollama run muse-glimmer:30b-mlx
TagSizeContext
muse-glimmer:latest18 GB128K
muse-glimmer:30b18 GB128K
muse-glimmer:30b-mlx21 GB128K

(Source: ollama.com/library/muse-glimmer, retrieved 2026-08-11.)

Ollama exposes an OpenAI-compatible endpoint at http://localhost:11434/v1 once the model is loaded, which is enough for most agent frameworks. What it does not give you easily is fine control over sampling and the vision projector — for that, drop to llama.cpp.

3. The controllable path: llama.cpp

llama.cpp can pull straight from Hugging Face, so you do not need to download files manually:

./llama.cpp/llama-cli \
    -hf unsloth/Muse-Glimmer-30B-GGUF:UD-Q4_K_XL \
    --temp 1.0 \
    --top-p 0.95 \
    --top-k 64

Those three sampling values are not arbitrary defaults — they are the parameters Meta recommends and Unsloth reproduces. A temperature of 1.0 is unusually high for an agentic model, and lowering it to the 0.2–0.3 you might use with other coding models will measurably hurt tool-call diversity. Leave it at 1.0 unless you have a specific reason.

For vision, point at both files:

./llama.cpp/llama-cli \
    --model unsloth/Muse-Glimmer-30B-GGUF/Muse-Glimmer-30B-UD-Q4_K_XL.gguf \
    --mmproj unsloth/Muse-Glimmer-30B-GGUF/mmproj-BF16.gguf \
    --temp 1.0 \
    --top-p 0.95 \
    --top-k 64

And to serve it as an OpenAI-compatible API — this is the form you actually want for agent work:

./llama.cpp/llama-server \
    --model unsloth/Muse-Glimmer-30B-GGUF/Muse-Glimmer-30B-UD-Q4_K_XL.gguf \
    --mmproj unsloth/Muse-Glimmer-30B-GGUF/mmproj-BF16.gguf \
    --temp 1.0 \
    --top-p 0.95 \
    --top-k 64 \
    --alias "unsloth-Muse-Glimmer-30B-GGUF" \
    --port 8001

(All three commands quoted verbatim from Unsloth’s Muse Glimmer docs, retrieved 2026-08-11.)

If you have hit KV-cache reuse problems with llama-server before, our llama-server KV cache fix guide applies here unchanged — nothing about Muse Glimmer’s architecture changes that behaviour.

4. The control most guides miss: reasoning strength

Muse Glimmer exposes four reasoning effort levels — low, medium, high, xhigh — and this is the single highest-leverage knob in the model.

Unsloth’s guidance is to use high or xhigh for complex problem-solving, coding, and agentic tasks. Meta’s own inference examples in the Hugging Face launch post pass it as a parameter:

reasoning_strength='low'

(Source: Hugging Face launch post, retrieved 2026-08-11.)

Unsloth’s docs describe it as a system-prompt directive of the form Reasoning strength: <value>. These are two different interfaces and it is not currently clear from public docs which one applies to which runtime — the parameter form appears in transformers examples, the system-prompt form in the GGUF context. If you are on llama.cpp, try the system-prompt form first and verify the model is actually producing longer reasoning traces before assuming it took effect.

This matters economically: on a local model, reasoning strength is the dial between “answers in 3 seconds” and “answers in 40 seconds but actually completes the task.” For an always-on agent, running everything at xhigh will make your machine unusable. Route by task.

5. Tool calling

Meta’s official example uses the standard OpenAI function schema:

{
  "type": "function",
  "function": {
    "name": "weather.get",
    "description": "Get the current weather for a city.",
    "parameters": {
      "type": "object",
      "properties": {"city": {"type": "string"}},
      "required": ["city"]
    }
  }
}

(Source: Hugging Face launch post, retrieved 2026-08-11.)

This is the part of the model that justifies choosing it. On MCP Atlas, Muse Glimmer scores 75.5 against Qwen3.6-27B’s 62.5 and Gemma4-31B’s 54.2 — a 13-point lead over the nearest competitor on the benchmark that specifically measures picking the right tool with the right arguments from a large registry.

On vLLM, tool calling may need an explicit parser flag — and the exact name is unconfirmed. Third-party write-ups mention --enable-auto-tool-choice --tool-call-parser muse_glimmer, and warn that without it tool calls surface as broken markup rather than structured calls. Meta’s own vLLM documentation does not show this flag — it says only to “use the release-matched Muse Glimmer parser package for native ATEM tool calls.” Treat the exact flag name as unconfirmed until you check the release-matched recipe for your vLLM version; do not copy it blind.

6. Serving it properly: vLLM and SGLang

Meta’s documented vLLM command is deliberately minimal:

python -m vllm.entrypoints.openai.api_server \
  --model meta-models/Muse-Glimmer-30B \
  --tensor-parallel-size 1 \
  --max-model-len 16384 \
  --port 8000

(Source: dev.meta.ai vLLM guide, retrieved 2026-08-11.)

Note --max-model-len 16384 — that is a conservative starting point, not the model’s limit. Raise it toward 131072 only as your KV cache budget allows.

Unsloth’s NVFP4 path is shorter and adds speculative decoding:

vllm serve unsloth/Muse-Glimmer-30B-NVFP4
vllm serve unsloth/Muse-Glimmer-30B-NVFP4 \
    --speculative-config '{"method": "mtp", "num_speculative_tokens": 2}'

SGLang:

python -m sglang.launch_server --model-path unsloth/Muse-Glimmer-30B-NVFP4 \
    --speculative-algorithm NEXTN \
    --speculative-num-steps 3 --speculative-eagle-topk 1 --speculative-num-draft-tokens 4

7. DFlash speculative decoding: read the table, not the headline

Meta ships a lightweight drafter alongside the model. Its specification, from the model card:

  • 5 draft layers, block size 16 tokens per forward pass
  • Sliding-window attention 2,048; 32 query heads / 8 KV heads (GQA)
  • Hidden features drawn from target layers 1, 13, 25, 37, 49 of 52
  • Sequence length 131,072

The published speedups:

HardwareBaselineWith DFlashSpeedup
NVIDIA RTX 509074.9 tok/s233.4 tok/s3.1x
Apple M5 Max26.6 tok/s50.2 tok/s1.8x
Apple M4 Max23.7 tok/s37.8 tok/s1.5x

(Source: Muse Glimmer-30B model card, retrieved 2026-08-11.)

Plan from your own row, not the headline. The 3.1x figure gets quoted everywhere; it is a CUDA number. Speculative decoding trades extra compute (verifying 16 speculative tokens in parallel) for fewer sequential steps. On a memory-bandwidth-bound Apple machine you have less spare compute to trade, so you collect 1.5–1.8x. That is still worth having — it is roughly the difference between a local agent feeling sluggish and feeling usable — but if you sized your expectations on 3.1x you will be disappointed.

8. Where Muse Glimmer is actually worse

This is the section most launch coverage skips, and Meta publishes the numbers itself.

BenchmarkMuse Glimmer-30BQwen3.6-27B
SWE-Bench Verified76.077.2
TerminalBench 2.1 (terminus2)51.760.7
OSWorld-Verified65.975.6
SkillsBench (with skills)44.346.6
GDPVal-AA v29531141
ScreenSpot Pro75.476.1
OmniDocBench v1.575.877.8

(Source: Muse Glimmer-30B model card benchmark table, both models in thinking / high-reasoning mode, retrieved 2026-08-11.)

The pattern is consistent: Muse Glimmer picks tools better; Qwen3.6-27B does the work better. A 9-point TerminalBench gap and a 10-point OSWorld gap are not noise. If your agent’s job is to drive a terminal or a GUI, Qwen3.6-27B is the stronger choice today — at 3B fewer parameters.

Choose Muse Glimmer when your bottleneck is orchestration: many tools, many MCP servers, long-horizon routing, ambiguous instructions. Choose Qwen3.6-27B when your bottleneck is execution. Our Qwen 3.6 local coding guide covers that setup.

9. The security numbers are the underrated reason to pick it

For a model that will sit resident on your machine reading untrusted content, these two rows matter more than any coding benchmark:

Metric (lower is better)Muse GlimmerGemma4-31BQwen3.6-27B
Siren AgentDojo — attack success rate28.4%25.6%40.3%
CI Memories — privacy violation rate26.4%12.1%53.4%

Utility stays level (AgentDojo utility 94.2 for Muse Glimmer vs 92.7 for Qwen), so the injection resistance is not bought with capability. Against Qwen3.6-27B, Muse Glimmer roughly halves both the prompt-injection success rate and the privacy violation rate. Gemma4-31B is better still on both — but gives up 13–21 points of agentic capability to get there.

If your local agent browses the web or reads your inbox, weight this table above SWE-Bench.

10. Gotchas

1. The vision projector is a separate file. mmproj-BF16.gguf is not bundled. Omit it and images are silently ignored rather than erroring.

2. Do not lower the temperature out of habit. 1.0 / 0.95 / 64 is the recommended configuration. Coding-model instincts (temp 0.2) will hurt here.

3. Image-heavy loops will be slower than you expect. The vision encoder emits up to 4,096 visual tokens per image against a 2,048-token sliding window, so every image pushes work onto the global-attention layers — the expensive one in four.

4. --max-model-len 16384 in the official vLLM command is not the model’s limit. It is a safe default. Raise it deliberately.

5. Reasoning strength has two apparent interfaces. Verify it took effect by checking trace length rather than assuming.

6. The chem/bio preparedness scores are below Gemma4-31B on 4 of 6 evaluations. Meta presents this as a safety result. Read it as evidence the model was tuned rather than purely scaled — relevant if you were expecting uniform superiority.

When not to use it

  • You need the best local coder. Qwen3.6-27B wins SWE-Bench Verified, TerminalBench and OSWorld at a smaller size.
  • You have under 12 GB of VRAM. The IQ2 quants exist but degrade meaningfully; a smaller model at higher precision will serve you better.
  • You need a permissive license and nothing else. Several open-weight models now ship Apache 2.0; that alone is no longer a differentiator.
  • Your workload is single-turn chat. This model’s advantages are all in multi-step tool loops. For chat you are paying 30B of parameters for capabilities you will not exercise.

Sources