WHAT IT IS
DBRX is an open-weights language model released by Databricks in 2024, using a mixture-of-experts (MoE) architecture. It has about 132 billion total parameters with roughly 36 billion active per forward pass. It targets enterprises that want to host and customise a large model in their own environment.
Why it matters
With an MoE architecture released as open weights, it let enterprises self-host a customisable large model in their own environment.
Key specs
- Architecture
- Mixture of experts (MoE)
- Total parameters
- About 132 billion
- Active parameters
- About 36 billion
- Context
- 32K tokens
Capabilities
Related concepts
Transformer Architecture
Replacing word-by-word relay with a room where everyone speaks at once, so long-range dependencies are one hop away
Pretraining & Fine-tuning
Learn language first from vast unlabelled text, then specialise with little data — the most data-efficient paradigm in modern AI
Comparable products
Llama
2023The model family that made the open-weight route mainstream
Qwen
2023An open-weight family spanning many sizes, with multimodal versions
Mistral Large
2024The flagship commercial model from a European open-weights lab