Production model

DeepSeek R1

DeepSeek-AI · Released 2025-01-20 · 671B parameters (37B active) · 128,000 ctx · open weights

Architecture

Positional encoding	RoPE (decoupled head for MLA)
Norm placement	Pre-Norm
Norm type	RMSNorm
QK-Norm	no
Activation	SwiGLU
Attention	MLA
MoE	DeepSeekMoE with aux-loss-free routing
Other	Same base architecture as DeepSeek V3 (61 layers, 128 attention heads, 256 routed experts + 1 shared), Two-stage post-training: cold-start SFT, then RL with GRPO and rule-based reward, Distilled smaller variants (1.5B–70B) released alongside the 671B

DeepSeek R1 (January 2025) is the reasoning-trained variant of DeepSeek V3. Architecturally it is the V3 base model unchanged: 671B-parameter MoE with 37B active per token, 61 layers, MLA attention with a decoupled RoPE head, DeepSeekMoE with aux-loss-free routing, Pre-Norm RMSNorm, SwiGLU.

What R1 contributes is post-training, not architecture:

Cold-start supervised fine-tuning on a small set of high-quality reasoning traces — long chain-of-thought completions with explicit <think> and <answer> tags.
Reinforcement learning with GRPO (Group Relative Policy Optimization, a PPO variant the DeepSeek-Math team introduced in 2024) using rule-based rewards: correctness checked by programmatic verifiers (math answer equality, code unit-test pass), not a learned reward model. The paper’s central observation is that long chain-of-thought emerges spontaneously from RL on rule-checkable problems, without explicit reasoning-trace supervision.
Distillation of R1’s reasoning behavior into smaller dense bases (Qwen, Llama derivatives at 1.5B–70B), released as DeepSeek-R1-Distill-* checkpoints.

For the architecture details that R1 inherits unchanged from V3 — MLA’s KV-cache compression, DeepSeekMoE’s shared-expert routing, the aux-loss-free balance bias, RoPE+YaRN extension to 128K — see the DeepSeek V3 entry.

This page exists for navigability: R1 is the most-cited open reasoning model of early 2025, and readers arriving from “what architecture does R1 use?” should land somewhere useful. The architecture answer is “exactly V3.”

Sources

Export

BibTeX

@article{arxiv_2501_12948,
  title         = {DeepSeek R1},
  author        = {DeepSeek-AI},
  year          = {2025},
  eprint        = {2501.12948},
  archivePrefix = {arXiv},
  url           = {https://arxiv.org/abs/2501.12948}
}

CSL JSON

{
  "id": "arxiv_2501_12948",
  "type": "article-journal",
  "title": "DeepSeek R1",
  "author": [
    {
      "literal": "DeepSeek-AI"
    }
  ],
  "issued": {
    "date-parts": [
      [
        2025
      ]
    ]
  },
  "URL": "https://arxiv.org/abs/2501.12948",
  "number": "2501.12948",
  "source": "arXiv"
}

RIS

TY  - JOUR
TI  - DeepSeek R1
AU  - DeepSeek-AI
PY  - 2025
JO  - arXiv
AN  - arXiv:2501.12948
UR  - https://arxiv.org/abs/2501.12948
ER  -