Jul 6, 2026

How to Run GLM-5.2 Locally (the MIT 744B Coder, and the 2-bit Build That Fits a 256GB Mac)

Read time: ~8 minutes. What you’ll learn: what GLM-5.2 is and why an MIT-licensed frontier coder matters, its real benchmark numbers, the honest hardware math for a 744B model (the smallest usable quant is ~239 GB — but that 2-bit build actually fits a 256 GB Mac or a 24 GB GPU with offload), and three real ways to run it — llama.cpp + Unsloth GGUF for big-memory local rigs, the official vLLM / SGLang servers for GPU boxes, and the Z.ai API for everyone else — plus the thinking-effort controls, context settings, and the sampling values from the official card.

Sourcing note: the architecture, deployment frameworks (with exact versions), benchmarks, and sampling settings are quoted from the official zai-org/GLM-5.2 model card and its config.json. The GGUF quant sizes, memory guidance, and the llama.cpp command are from the Unsloth GLM-5.2 docs — Unsloth’s GGUF is a third-party (but widely used) distribution, flagged as such. Benchmark numbers are Z.ai’s own. Links at the bottom.

Most open-weight coding models make you choose: frontier quality or a license you can actually ship on. GLM-5.2 is Z.ai’s answer to “why not both” — a flagship agentic-coding model released under a plain MIT license, no regional limits, that trades blows with Claude Opus 4.8 and GPT-5.5 on coding and agent benchmarks. If you want a top-tier coder you can self-host and build a product on without license anxiety, this is the one to run. Here’s how — honestly, including when not to run it locally.

1. What GLM-5.2 actually is

GLM-5.2 is Z.ai’s latest flagship, tuned for long-horizon coding and agentic work. From the official model card and config.json:

744B total parameters, 40B active — a Mixture-of-Experts model with 256 routed experts (8 active per token) + 1 shared expert, 78 layers (the first 3 are dense), built on the new GlmMoeDsa (deep sparse-attention MoE) architecture.
A solid 1M-token context (1,048,576 positions) — Z.ai stresses the “solid” part: the context stably sustains long-horizon work rather than degrading at the tail.
License: MIT — open weights, commercially usable, no regional restrictions. This is the headline: an MIT frontier coder is rare.
Flexible thinking effort — GLM-5.2 has multiple reasoning-effort levels (and a no-think mode) so you can trade latency for depth per request, not per deployment.

Two architecture notes Z.ai calls out. IndexShare reuses the same attention indexer across every four sparse-attention layers, cutting per-token FLOPs by 2.9× at 1M context — that’s what makes the million-token window practical instead of ruinously slow. And an improved MTP layer (multi-token prediction) raises the acceptance length for speculative decoding by up to 20%, which matters if you serve it with a framework that exploits MTP.

The “40B active” part is what makes a 744B model tractable at inference: only 40B parameters fire per token. But — and this is the crux of §3 — MoE cuts compute, not memory. You still have to hold (nearly) all 744B parameters in memory to serve the model.

2. What it can actually do

Z.ai’s reported results (their own numbers, pending independent reproduction), against the field:

Benchmark	GLM-5.2	GLM-5.1	DeepSeek-V4-Pro	Claude Opus 4.8	GPT-5.5
SWE-bench Pro (coding)	62.1	58.4	55.4	69.2	58.6
Terminal-Bench 2.1 (Terminus-2)	81.0	63.5	64	85	84
NL2Repo	48.9	42.7	35.5	69.7	50.7
MCP-Atlas (agentic, public set)	76.8	71.8	73.6	77.8	75.3
AIME 2026 (math)	99.2	95.3	94.6	95.7	98.3
HLE (reasoning)	40.5	31	37.7	49.8	41.4

Read it honestly: GLM-5.2 doesn’t beat Claude Opus 4.8 on raw coding (62.1 vs 69.2 on SWE-bench Pro), but it clears GPT-5.5 on SWE-bench Pro (62.1 vs 58.6), essentially ties the frontier on agentic tool use (MCP-Atlas 76.8 vs Opus’s 77.8), and posts a large generational jump over GLM-5.1 nearly everywhere (Terminal-Bench 63.5 → 81.0). For an MIT-licensed, self-hostable model, landing in the same bracket as the closed frontier on agentic coding is the whole story. If you’re building an agent and license terms decide whether you can ship, GLM-5.2 is the strongest open option in this table.

3. The hardware reality check (read before downloading 239 GB)

Here’s the honest math. Even heavily quantized, a 744B model is large. From Unsloth’s GGUF builds:

Quant	Realistic RAM + VRAM needed
UD-IQ1_S (1-bit)	~223 GB
UD-IQ2_M (2-bit)	~245 GB (239 GB on disk)
3-bit	~290–360 GB
4-bit	~372–475 GB
5-bit	~570 GB
8-bit	~810 GB

The rule of thumb: your combined RAM + VRAM should roughly cover the quant’s memory figure. So the smallest genuinely usable build (2-bit) wants ~239–245 GB.

But here’s where GLM-5.2 is friendlier than a 1T model like Kimi K2.7: Unsloth notes the 2-bit quant “can directly fit on a 256 GB unified-memory Mac,” and also runs on a single 24 GB GPU + 256 GB system RAM with MoE offloading (the active-expert weights live on the GPU, the rest stream from RAM). That’s still a serious workstation — a 256 GB Mac Studio or a big-RAM DDR5 desktop with one prosumer GPU — but it’s an attainable single machine, not a multi-GPU server rack. That distinction is the reason this model is worth a local guide at all.

Bottom line:

256 GB unified-memory Mac, or a 24 GB GPU + 256 GB RAM? §4 (llama.cpp) is for you.
Multi-GPU server at higher precision? §5 (vLLM / SGLang).
A normal machine — even a very good one? §6. Use the Z.ai API. This guide will tell you so honestly rather than pretend 744B fits in 24 GB of VRAM alone.

4. Method 1 — llama.cpp + Unsloth GGUF (big-memory local)

If you have the memory, the local route is Unsloth’s GGUF quants run through llama.cpp. Note the Unsloth GGUF is a third-party distribution (widely used, but not Z.ai’s own upload) — call it out if provenance matters to your pipeline.

The recommended command, quoted from the Unsloth docs:

./llama.cpp/llama-cli \
    --model unsloth/GLM-5.2-GGUF/UD-IQ2_M/GLM-5.2-UD-IQ2_M-00001-of-00006.gguf \
    --temp 1.0 --top-p 0.95 --min-p 0.01

What to know:

The model ships as a multi-part GGUF (00001-of-00006 here) — download all shards; llama.cpp loads the set from the first file.
Recommended sampling: --temp 1.0 --top-p 0.95 --min-p 0.01 — these match the model card’s evaluation settings; use them.
On a 256 GB Mac, the 2-bit UD-IQ2_M fits in unified memory directly. On a 24 GB GPU, add MoE offload so the non-active experts stream from system RAM (you need ~256 GB RAM for that path).
To serve an API instead of a CLI, use llama-server from the same GGUF and hit it as an OpenAI-compatible endpoint — that’s how you point a coding agent at it.

Controlling the thinking effort

GLM-5.2’s flexible reasoning is exposed through the chat template. From the Unsloth docs:

# turn thinking off (fast, cheap)
--chat-template-kwargs '{"enable_thinking":false}'

# crank reasoning up for hard problems
--chat-template-kwargs '{"reasoning_effort":"high"}'
--chat-template-kwargs '{"reasoning_effort":"max"}'

Use max/high for genuinely hard agentic or math tasks and false for quick edits — that per-request control is one of GLM-5.2’s most practical features.

5. Method 2 — Official vLLM / SGLang (GPU servers)

If you have the GPU memory to serve the weights at higher precision, GLM-5.2 has official framework support. These pull the real weights (zai-org/GLM-5.2), not a repackage. Mind the minimum versions — GLM-5.2 uses the new glm_moe_dsa architecture, so older builds won’t load it:

SGLang — v0.5.13.post1+ (official cookbook)
vLLM — v0.23.0+ (official recipe)
Transformers — v0.5.12+ (model docs)
KTransformers — v0.5.12+ (built for exactly the GPU-plus-RAM-offload case, if you want offload on the official stack rather than llama.cpp)
Ascend NPU — vLLM-Ascend / xLLM / SGLang are supported

Both vLLM and SGLang expose an OpenAI-compatible endpoint you point your agent at. This is the route for a team self-hosting for data control or throughput at usable precision (4-bit and up wants 372 GB+ of memory — plan capacity before you commit). If your framework supports GLM-5.2’s MTP speculative decoding, enable it: the model card’s improved MTP is worth up to a 20% acceptance-length gain.

6. Method 3 — The pragmatic path: the Z.ai API

For most builders, this is the right starting point, and there’s no shame in it — you don’t self-host 744B to evaluate a model.

Z.ai API Platform. GLM-5.2 is served at docs.z.ai with an OpenAI-compatible endpoint — swap your base URL and model name and most existing tooling just works.
chat.z.ai. Want to feel it out before writing a line of code? It’s live in the browser at chat.z.ai.

The honest trade: local gives you data control, an MIT license you own, and no per-token bill; the API gives you full-quality inference, zero hardware outlay, and you’re running in minutes. Unless you specifically need on-prem or already own the 256 GB machine, start with the API, confirm GLM-5.2 fits your workload, then decide whether self-hosting is worth the memory.

7. Who this is actually for

Be honest about which bucket you’re in:

Self-host locally (Method 1/2) if you already have a 256 GB Mac / big-RAM + GPU rig or a GPU server, you need on-prem data control, or your volume makes per-token API cost dominate hardware amortization. For those, the weights being MIT and openly available is exactly the point — this is the rare frontier-class coder you can legally build a commercial product on top of, on your own hardware.
Use the Z.ai API (Method 3) if you’re evaluating, iterating, or running normal volumes on normal hardware — which is almost everyone. Full-quality GLM-5.2, no memory bill.

The failure mode to avoid: downloading 239 GB, discovering it swaps to disk at a token per second on an under-memoried box, and concluding the model is bad — when you just picked the wrong path for your machine. Match the route to your hardware.

The takeaway

GLM-5.2 is a genuinely strong MIT-licensed, 744B-total / 40B-active open-weight coder — 1M context, flexible thinking effort, 62.1 on SWE-bench Pro and 81.0 on Terminal-Bench 2.1, in the same bracket as the closed frontier on agentic tool use. Running it locally is real but heavy: the smallest usable quant is ~239–245 GB, via Unsloth’s GGUF + llama.cpp — and unlike a 1T model, that 2-bit build actually fits a 256 GB Mac or a 24 GB GPU with offload. For higher precision, use the official vLLM / SGLang servers; for everyone else, the Z.ai API is the honest best path. Start on the API to validate the model, and take on 239 GB of memory only once you’re sure it’s worth it — then enjoy a frontier coder whose license is finally on your side.

For other local coding models across the size spectrum, see Kimi K2.7 Code locally (1T, the bigger sibling), Qwen 3.6 for local coding, and Ornith-1.0 locally (9B, genuinely laptop-friendly).

Sources

zai-org/GLM-5.2 — Hugging Face model card — MIT license, flagship long-horizon coder, solid 1M context, flexible thinking effort, IndexShare (−2.9× per-token FLOPs at 1M) + improved MTP (up to +20% speculative-decoding acceptance), benchmarks (SWE-bench Pro 62.1, Terminal-Bench 2.1 Terminus-2 81.0, NL2Repo 48.9, MCP-Atlas 76.8, AIME 2026 99.2, HLE 40.5), deployment frameworks with minimum versions (SGLang v0.5.13.post1+, vLLM v0.23.0+, Transformers v0.5.12+, KTransformers v0.5.12+, Unsloth v0.1.47-beta+, Ascend NPU), sampling temperature 1.0 / top_p 0.95, Z.ai API + chat.z.ai
GLM-5.2 config.json (official) — GlmMoeDsaForCausalLM, 78 layers, 256 routed experts + 1 shared, 8 experts per token, first 3 layers dense, hidden size 6144, vocab 154,880, 1,048,576 max positions
GLM-5.2 — Unsloth docs (Run Locally) — 744B total / 40B active; GGUF memory figures (IQ1_S ~223 GB, IQ2_M ~239 GB disk / ~245 GB RAM+VRAM, 3-bit ~290–360 GB, 4-bit ~372–475 GB, 5-bit ~570 GB, 8-bit ~810 GB); 2-bit fits a 256 GB unified-memory Mac or a 24 GB GPU + 256 GB RAM with MoE offload; llama.cpp command with --temp 1.0 --top-p 0.95 --min-p 0.01; thinking controls via --chat-template-kwargs. Third-party GGUF distribution.
Benchmark numbers are Z.ai’s own reported results, pending independent reproduction. Hardware figures are Unsloth’s guidance. Verified July 6, 2026 — confirm current details on the official model card and Unsloth docs before relying on them.