Facets Routing Mechanisms that send tokens to a subset of compute. 7 techniques tagged with this facet. FFN & MoE Sparsely-Gated MoE Sparse MoE 2017-01 Top-K gating with per-expert noise plus an importance-balancing loss. The 2017 paper that established that 'sparse activation + load balance' is a viable training-time recipe; every modern MoE descends from it. GShard GShard 2020-06 Top-2 routing across hundreds of experts in a sharded transformer encoder. Established expert capacity, dispatch tensors, and the per-expert random-token-dropping policy. The conceptual ancestor of Switch, Mixtral, and DeepSeekMoE. Switch Transformer Switch 2021-01 Top-1 expert routing — each token goes to exactly one expert. Simpler than GShard's top-2 baseline. Trillion-parameter Switch-C model proved sparse MoE was tractable at unprecedented scale; established the load-balancing loss formulation everyone uses. DeepSeekMoE DeepSeekMoE 2024-01 Split the FFN into many small experts, route each token to top-K, and reserve a few always-on shared experts for common-knowledge work. Activated parameter count stays small; specialization sharpens. Mixtral-Style Coarse MoE Mixtral MoE 2024-01 Eight full-sized SwiGLU experts per MoE layer; top-2 routing with a standard load-balancing auxiliary loss. The Mistral-org variant that brought open-weights MoE into wide use, sitting on the coarse end of the granularity spectrum that DeepSeekMoE later contested. Auxiliary-Loss-Free Load Balancing Aux-Loss-Free 2024-08 Maintain a per-expert bias term that shifts the gate's top-K decision toward under-used experts. The bias enters at selection time only — the final expert outputs are weighted by the original (unbiased) gate values. Load balance without quality cost. Kimi K2 MoE K2 MoE 2025-07 Trillion-parameter MoE with 384 routed experts plus one shared expert — wider than DeepSeek V3's 256 + 1 by ~50%. Stabilized at scale by the MuonClip optimizer (Muon family with weight clipping). Same shared-expert + aux-loss-free routing lineage as DeepSeekMoE.