입력텍스트텍스트
이 페이지의 본문은 영어로 제공됩니다. 제목과 요약은 한국어로 번역되었습니다.
무엇인가
Jamba is an open-weight model AI21 Labs released in March 2024, interleaving Mamba state-space layers with Transformer attention layers inside one mixture-of-experts architecture. AI21 Labs was founded in Tel Aviv in 2017 by Amnon Shashua and others. This hybrid design holds long context while cutting memory and compute relative to a pure-attention model. Jamba ships as a 52B-parameter MoE that activates about 12B per token, with a context window in the 256K range.
기억할 만한 이유
It replaced most attention layers with state-space layers, testing whether a non-pure-Transformer architecture can save compute at long context — a concrete instance of hybrid architectures entering open-weight models.
주요 사양
- Parameters
- 52B (MoE, ~12B active)
- Context window
- 256K tokens
- Architecture
- Mamba state-space layers interleaved with Transformer layers
- Open weights
- Yes