Open Mixture-of-Experts models share a core architectural principle but diverge significantly in their token routing mechanisms. A comparative review of ten prominent models reveals wide variations in expert counts, active experts per token, and load-balancing strategies.

Key Developments and Technical Context

Expert counts across open Mixture-of-Experts models range from 8 experts in Mixtral to 384 experts in Kimi K2 and DeepSeek-V4-Pro.

The top-k parameter determining experts per token now spans from 1 in Llama 4 Maverick to 8 in models such as DeepSeek-V3 and Kimi K2.

Most surveyed models maintain one always-on shared expert, while Mixtral, gpt-oss, and Qwen3 omit this component.

Strategic Analysis and Industry Implications

Variations in router design directly impact inter-device communication bandwidth and the hardware resource requirements needed to serve the model.

The omission of gating functions or balancing methods from configuration files in certain models complicates efforts to fully audit their architectural transparency.

Outlook and Key Factors to Watch

The precise motivations behind the omission of routing configuration details by certain developers remain unclarified.

The practical performance of these routing methods when scaled to multi-trillion parameters in large production environments requires further independent validation data.