Production model

Qwen3 235B-A22B

Alibaba (Qwen Team) · Released 2025-05-14 · 235B parameters (22B active) · 128,000 ctx · open weights

Architecture

Positional encoding	RoPE (base 10K → 1M via ABF; YaRN + DCA for long context)
Norm placement	Pre-Norm
Norm type	RMSNorm
QK-Norm	yes
Activation	SwiGLU
Attention	GQA (64 query heads, 4 KV heads)
MoE	Sparse MoE — 128 experts, top-8 routing, no shared expert, global-batch load balancing
Other	94 layers in the MoE flagship, Apache 2.0 license, Unified thinking / non-thinking modes via prompt template, Removes QKV bias (vs Qwen 2.5) and introduces QK-Norm for training stability

Techniques used

Grouped-Query Attention — 64 query heads over 4 KV heads (G = 16) in the 235B-A22B flagship; technical report §2 Table 1.
Sparsely-Gated MoE — 128 experts, top-8 routing, no shared expert; global-batch load balancing instead of per-batch aux loss.
Swish-Gated Linear Unit — SwiGLU FFN inside each routed expert (Qwen 3 technical report §2).
Query-Key Normalization — Qwen3 removes the QKV bias used in Qwen 2.5 and adds QK-Norm for training stability at scale (technical report §2.1).
Root Mean Square Layer Normalization — Pre-Norm RMSNorm across the 94-layer MoE flagship.
Rotary Position Embedding — RoPE base scaled from 10K to 1M via ABF; YaRN + DCA for long-context extension.
YaRN — Yet Another RoPE eXtensioN — YaRN + Dual Chunk Attention (DCA) for context extension beyond the ABF-trained 32K window.

Qwen3 235B-A22B (May 2025) is Alibaba’s flagship MoE in the Qwen 3 generation: 235B total parameters, 22B active per token. Architecturally it stays close to the consensus open-weights stack, with two deliberate departures from Qwen 2.5:

QK-Norm. Qwen 3 drops the QKV bias used in Qwen 2.5 and adds QK-Norm to the attention mechanism for training stability at scale. The technical report (§2.1) attributes the change to stability issues seen at the 235B scale.
No shared expert, global-batch load balancing. Unlike Qwen 2.5-MoE and the DeepSeek-MoE-derived families, Qwen 3 MoE drops the always-on shared expert and uses a global-batch load-balancing loss to encourage expert specialization.

The rest of the stack follows the dominant 2024–25 pattern: Pre-Norm RMSNorm, GQA with 64 query heads sharing 4 KV heads, SwiGLU FFN inside each MoE expert, RoPE with the base scaled from 10K to 1M via ABF for long-context training, plus YaRN and Dual Chunk Attention (DCA) for sequence-length extension at inference.

The unified thinking / non-thinking inference mode — a single model that produces internal reasoning traces when asked, or responds directly otherwise — is selected at the prompt template level rather than via a separate model variant.

For the dense flagship in the same generation see Qwen3 32B; for the small MoE see Qwen3 30B-A3B; for the hybrid-attention follow-up see Qwen3-Next 80B-A3B.

Sources

Export

BibTeX

@article{arxiv_2505_09388,
  title         = {Qwen3 235B-A22B},
  author        = {Alibaba (Qwen Team)},
  year          = {2025},
  eprint        = {2505.09388},
  archivePrefix = {arXiv},
  url           = {https://arxiv.org/abs/2505.09388}
}

CSL JSON

{
  "id": "arxiv_2505_09388",
  "type": "article-journal",
  "title": "Qwen3 235B-A22B",
  "author": [
    {
      "literal": "Alibaba (Qwen Team)"
    }
  ],
  "issued": {
    "date-parts": [
      [
        2025
      ]
    ]
  },
  "URL": "https://arxiv.org/abs/2505.09388",
  "number": "2505.09388",
  "source": "arXiv"
}

RIS

TY  - JOUR
TI  - Qwen3 235B-A22B
AU  - Alibaba (Qwen Team)
PY  - 2025
JO  - arXiv
AN  - arXiv:2505.09388
UR  - https://arxiv.org/abs/2505.09388
ER  -

Models like this

Qwen3 30B-A3BAlibaba (Qwen Team)86% overlap
Qwen3 32BAlibaba (Qwen Team)86% overlap
OLMo 3 32BAllen Institute for AI (AI2)67% overlap