Production model

DeepSeek V3.1

DeepSeek-AI · Released 2025-08-21 · 671B parameters (37B active) · 128,000 ctx · open weights

Architecture

Positional encoding	RoPE (decoupled head for MLA)
Norm placement	Pre-Norm
Norm type	RMSNorm
QK-Norm	no
Activation	SwiGLU
Attention	MLA
MoE	DeepSeekMoE with aux-loss-free routing
Other	Same base architecture as DeepSeek V3 / R1 (671B total, 37B active, 61 layers), Hybrid thinking + non-thinking modes selected by chat template, Continued pretraining and post-training on top of the V3 base

Techniques used

Multi-Head Latent Attention — Same MLA design carried over from V3.
Auxiliary-Loss-Free Load Balancing — Same aux-loss-free routing inherited from V3 across the MoE layers.

DeepSeek V3.1 (August 2025) is a continuation of the DeepSeek V3 base model with additional pretraining and a post-training recipe that supports both reasoning (“thinking”) and direct (“non-thinking”) inference modes from a single checkpoint. Architecturally it inherits the V3 stack unchanged:

671B-parameter MoE, 37B active per token, 61 layers
MLA attention with a decoupled RoPE head
DeepSeekMoE with aux-loss-free routing
Pre-Norm RMSNorm, SwiGLU FFN
128K context window via YaRN extension

The V3.1 contribution is at the post-training and weight level — extended pretraining, unified thinking/non-thinking distillation, refined tool-use behavior — not at the architecture level. For the architecture, the V3 entry is the source of truth.

This page exists for navigability. Readers landing here from “what does V3.1 use for attention?” should know the answer is “the same MLA configuration as V3.”

The DeepSeek V3.2-Exp release (when added) is where the architecture line resumes — V3.2 introduces DeepSeek Sparse Attention on top of MLA.

Sources

Models like this