Production model

GLM-4.5

Zhipu AI · Released 2025-07-28 · 355B parameters (32B active) · 131,072 ctx · open weights

Architecture

Positional encoding	RoPE (theta 1e6; partial rotary factor 0.5)
Norm placement	Pre-Norm
Norm type	RMSNorm
QK-Norm	yes
Activation	SwiGLU (SiLU)
Attention	GQA (96 query heads, 8 KV heads)
MoE	Sparse MoE — 160 routed experts, top-8 routing, 1 shared expert; routed_scaling_factor 2.5
Other	92 layers, hidden dim 5120, expert intermediate dim 1536, Depth-over-width: 92 layers with hidden dim 5120 (vs ~64 layers at hidden 7K for similar-active-param models), Multi-Token Prediction (MTP) head, Muon optimizer for training, MIT license

Techniques used

Grouped-Query Attention — 96 query heads over 8 KV heads (G = 12) — a high-Q / aggressive-GQA configuration.
DeepSeekMoE — 160 routed experts + 1 shared expert per MoE layer; top-8 routing; the shared-expert pattern carried over from the DeepSeek-MoE line.
Swish-Gated Linear Unit — SwiGLU (SiLU activation per the released config.json) inside each expert.
Query-Key Normalization — QK-Norm called out as one of the explicit architectural choices alongside GQA, MTP, and the Muon optimizer.
Root Mean Square Layer Normalization — RMSNorm with rms_norm_eps 1e-5 in the released config; 92 transformer layers.
Rotary Position Embedding — RoPE with theta = 10⁶ and a partial rotary factor of 0.5 — half the head dim carries rotation.

GLM-4.5 (July 2025) is Zhipu AI’s open-weights MoE flagship — 355B total parameters, 32B active per token, 92 transformer layers. The technical report (arXiv:2508.06471) and the released config.json together document a Pre-Norm RMSNorm + RoPE + GQA + SwiGLU

MoE stack with two notable choices:

Depth over width. 92 layers at hidden dim 5120 (vs the more common ~60–65 layers at hidden dim 7K–8K for similar active-parameter MoEs). The report frames the choice as a bet that depth improves reasoning at fixed active-param budget.
96 attention heads, 8 KV heads. A high query-head count — large query bandwidth per layer — combined with aggressive GQA sharing (G = 12).
QK-Norm in attention, plus Multi-Token Prediction (MTP) head for speculative decoding, and Muon as the pre-training optimizer.

The MoE is sized at 160 routed experts plus one always-on shared expert, top-8 routing, expert intermediate dim 1536. This puts the routing density (8 + 1 active per token) in between DeepSeek V3 (8 + 1 of 256 + 1) and Qwen3 235B (8 of 128, no shared expert).

A companion model, GLM-4.5-Air, uses the same architectural recipe at 106B total / 12B active for lower-cost deployment. Both ship under the MIT license.

Context handling at 128K uses RoPE with theta = 10⁶ and a partial rotary factor of 0.5 — half the head dim carries rotary position information, the other half is plain.

For Zhipu’s earlier dense entry see GLM-4 9B and the 2022 baseline GLM 130B.

Sources

Export

BibTeX

@article{arxiv_2508_06471,
  title         = {GLM-4.5},
  author        = {Zhipu AI},
  year          = {2025},
  eprint        = {2508.06471},
  archivePrefix = {arXiv},
  url           = {https://arxiv.org/abs/2508.06471}
}

CSL JSON

{
  "id": "arxiv_2508_06471",
  "type": "article-journal",
  "title": "GLM-4.5",
  "author": [
    {
      "literal": "Zhipu AI"
    }
  ],
  "issued": {
    "date-parts": [
      [
        2025
      ]
    ]
  },
  "URL": "https://arxiv.org/abs/2508.06471",
  "number": "2508.06471",
  "source": "arXiv"
}

RIS

TY  - JOUR
TI  - GLM-4.5
AU  - Zhipu AI
PY  - 2025
JO  - arXiv
AN  - arXiv:2508.06471
UR  - https://arxiv.org/abs/2508.06471
ER  -

Models like this

Hunyuan-Large 389BTencent83% overlap
Qwen3 30B-A3BAlibaba (Qwen Team)71% overlap
Qwen3 32BAlibaba (Qwen Team)71% overlap