Production model

Qwen3 30B-A3B

Alibaba (Qwen Team) · Released 2025-05-14 · 30B parameters (3B active) · 128,000 ctx · open weights

Architecture

Positional encoding	RoPE (base 10K → 1M via ABF; YaRN + DCA for long context)
Norm placement	Pre-Norm
Norm type	RMSNorm
QK-Norm	yes
Activation	SwiGLU
Attention	GQA (32 query heads, 4 KV heads)
MoE	Sparse MoE — 128 experts, top-8 routing, no shared expert, global-batch load balancing
Other	48 layers, Apache 2.0 license, 30B total, 3B active per token — small-MoE complement to the 235B-A22B flagship

Techniques used

Grouped-Query Attention — 32 query heads over 4 KV heads (G = 8) in the small MoE.
Sparsely-Gated MoE — Same 128-expert top-8 design as the Qwen 3 flagship at smaller hidden dim.
Swish-Gated Linear Unit — SwiGLU FFN inside each routed expert of the small MoE.
Query-Key Normalization — Carries QK-Norm from the Qwen 3 architectural pattern.
Root Mean Square Layer Normalization — Pre-Norm RMSNorm across the small MoE.
Rotary Position Embedding — RoPE with ABF-scaled base, shared across the Qwen 3 family.

Qwen3 30B-A3B (May 2025) is the small MoE variant of the Qwen 3 family — 30B total parameters, 3B active per token. It carries the same architectural pattern as the 235B-A22B flagship at a smaller scale: 48 layers, 32 query heads sharing 4 KV heads under GQA, Pre-Norm RMSNorm, SwiGLU FFN inside each expert, RoPE with ABF-scaled base, and QK-Norm in attention.

The MoE configuration is unchanged from the flagship: 128 experts, top-8 routing, no shared expert, global-batch load balancing. The intent — laid out in §2.1 of the technical report — is that the same routing recipe can be carried across both MoE sizes without retuning.

3B active parameters at competitive quality is the headline efficiency claim of the small MoE: the model achieves quality that the technical report compares to the 32B dense at a fraction of the per-token compute.

Sources

Export

BibTeX

@article{arxiv_2505_09388,
  title         = {Qwen3 30B-A3B},
  author        = {Alibaba (Qwen Team)},
  year          = {2025},
  eprint        = {2505.09388},
  archivePrefix = {arXiv},
  url           = {https://arxiv.org/abs/2505.09388}
}

CSL JSON

{
  "id": "arxiv_2505_09388",
  "type": "article-journal",
  "title": "Qwen3 30B-A3B",
  "author": [
    {
      "literal": "Alibaba (Qwen Team)"
    }
  ],
  "issued": {
    "date-parts": [
      [
        2025
      ]
    ]
  },
  "URL": "https://arxiv.org/abs/2505.09388",
  "number": "2505.09388",
  "source": "arXiv"
}

RIS

TY  - JOUR
TI  - Qwen3 30B-A3B
AU  - Alibaba (Qwen Team)
PY  - 2025
JO  - arXiv
AN  - arXiv:2505.09388
UR  - https://arxiv.org/abs/2505.09388
ER  -

Models like this

Qwen3 235B-A22BAlibaba (Qwen Team)86% overlap
GLM-4.5Zhipu AI71% overlap
Qwen3 32BAlibaba (Qwen Team)71% overlap