入力テキストテキスト
本ページの本文は英語で提供されています。タイトルと導入は日本語化されています。
これは何か
Jamba is an open-weight model AI21 Labs released in March 2024, interleaving Mamba state-space layers with Transformer attention layers inside one mixture-of-experts architecture. AI21 Labs was founded in Tel Aviv in 2017 by Amnon Shashua and others. This hybrid design holds long context while cutting memory and compute relative to a pure-attention model. Jamba ships as a 52B-parameter MoE that activates about 12B per token, with a context window in the 256K range.
なぜ覚えておく価値があるか
It replaced most attention layers with state-space layers, testing whether a non-pure-Transformer architecture can save compute at long context — a concrete instance of hybrid architectures entering open-weight models.
主な仕様
- Parameters
- 52B (MoE, ~12B active)
- Context window
- 256K tokens
- Architecture
- Mamba state-space layers interleaved with Transformer layers
- Open weights
- Yes