Production model

Qwen3 32B

Alibaba (Qwen Team) · Released 2025-05-14 · 32B parameters · 128,000 ctx · open weights

Architecture

Positional encoding	RoPE (base 10K → 1M via ABF; YaRN + DCA for long context)
Norm placement	Pre-Norm
Norm type	RMSNorm
QK-Norm	yes
Activation	SwiGLU
Attention	GQA (64 query heads, 8 KV heads)
MoE	—
Other	64 layers, Apache 2.0 license, Unified thinking / non-thinking modes via prompt template, Removes QKV bias (vs Qwen 2.5) and adds QK-Norm

Techniques used

Grouped-Query Attention — 64 query heads over 8 KV heads (G = 8) in the dense flagship.
Swish-Gated Linear Unit — SwiGLU FFN in the 64-layer dense flagship.
Query-Key Normalization — Same QK-Norm + no-QKV-bias change as the Qwen 3 MoE flagship.
Root Mean Square Layer Normalization — Pre-Norm RMSNorm across the 64-layer dense flagship.
Rotary Position Embedding — Same ABF + YaRN + DCA recipe as the Qwen 3 MoE flagship.
YaRN — Yet Another RoPE eXtensioN — Same YaRN + DCA extension recipe across the Qwen 3 family.

Qwen3 32B (May 2025) is the largest dense decoder in the Qwen 3 generation: 32B parameters, 64 layers, GQA with 64 query heads sharing 8 KV heads. The stack matches the Qwen 3 family pattern — Pre-Norm RMSNorm, SwiGLU FFN, RoPE with ABF-scaled base — plus the two Qwen 3 changes from Qwen 2.5:

QK-Norm replaces the QKV bias in attention for training stability.
Unified inference modes. Thinking and non-thinking outputs come from the same model, selected by prompt template; there is no separate “Instruct” vs “Reasoning” variant.

Context handling at 128K uses YaRN with Dual Chunk Attention (DCA) at inference time. Training data covers 119 languages (up from 29 in Qwen 2.5).

The dense 32B trades the MoE flagship’s quality-per-active-param efficiency for a simpler deployment footprint. For the MoE flagship in the same family see Qwen3 235B-A22B; for the small MoE see Qwen3 30B-A3B.

Sources

Export

BibTeX

@article{arxiv_2505_09388,
  title         = {Qwen3 32B},
  author        = {Alibaba (Qwen Team)},
  year          = {2025},
  eprint        = {2505.09388},
  archivePrefix = {arXiv},
  url           = {https://arxiv.org/abs/2505.09388}
}

CSL JSON

{
  "id": "arxiv_2505_09388",
  "type": "article-journal",
  "title": "Qwen3 32B",
  "author": [
    {
      "literal": "Alibaba (Qwen Team)"
    }
  ],
  "issued": {
    "date-parts": [
      [
        2025
      ]
    ]
  },
  "URL": "https://arxiv.org/abs/2505.09388",
  "number": "2505.09388",
  "source": "arXiv"
}

RIS

TY  - JOUR
TI  - Qwen3 32B
AU  - Alibaba (Qwen Team)
PY  - 2025
JO  - arXiv
AN  - arXiv:2505.09388
UR  - https://arxiv.org/abs/2505.09388
ER  -

Models like this

Qwen3 235B-A22BAlibaba (Qwen Team)86% overlap
OLMo 3 32BAllen Institute for AI (AI2)75% overlap
GLM-4.5Zhipu AI71% overlap