---
model-index:
- name: LFM2-8B-A1B — MLX (Apple Silicon), **3-bit** (with MoE + RAM planning)
  results: []
language:
- en
tags:
- mlx
- apple-silicon
- liquidai
- lfm2
- moe
- transformer
- long-context
- instruct
- quantized
- 3bit
- coding
pipeline_tag: text-generation
license: other
license_name: lfm1.0
license_link: LICENSE
library_name: mlx
base_model:
- LiquidAI/LFM2-8B-A1B
---

# LFM2-8B-A1B — **MLX 3-bit** (Apple Silicon)

**Maintainer / Publisher:** [**Susant Achary**](https://huggingface.co/Susant-Achary)  
**Upstream model:** [LiquidAI/LFM2-8B-A1B](https://huggingface.co/LiquidAI/LFM2-8B-A1B)  
**This repo (MLX 3-bit):** `mlx-community/LFM2-8B-A1B-3bit-MLX`

This repository provides an **Apple-Silicon-optimized MLX build** of **LFM2-8B-A1B** at **3-bit** quantization.  
3-bit is an excellent **size↔quality sweet spot** on many Macs—very small memory footprint with surprisingly solid answer quality and snappy decoding.

---

## 🔎 What is LFM2-8B-A1B?

- **Architecture:** Mixture-of-Experts (**MoE**) Transformer.  
- **Size:** ~**8B total parameters** with ~**1B active** per token (the “A1B” naming commonly indicates ~1B active params).  
- **Why MoE?** Per token, only a subset of experts is activated → **lower compute per token** while retaining a larger parameter pool for expressivity.

> **Memory reality on a single device:** Even though ~1B parameters are *active* at a time, **all experts typically reside in memory** in single-device runs. Plan **RAM** based on **total parameters**, not just the active slice.

---

## 📦 What’s in this MLX build

- `config.json` (MLX), `mlx_model*.safetensors` (**3-bit** shards)  
- Tokenizer: `tokenizer.json`, `tokenizer_config.json`  
- Metadata: `model_index.json` (and/or processor metadata as applicable)

Target: **macOS** on **Apple Silicon (M-series)** using **Metal/MPS**.

---

## ✅ Intended use

- General **instruction following**, chat, and summarization  
- **RAG** back-ends and long-context assistants on device  
- **Schema-guided** structured outputs (JSON) where low RAM is a priority

## ⚠️ Limitations

- 3-bit is **lossy**: tiny improvements in latency/RAM come with some accuracy trade-off vs 6/8-bit.  
- For very long contexts and/or batching, **KV-cache** can dominate memory—tune `max_tokens` and batch size.  
- Add your own **guardrails/safety** for production deployments.

---

## 🔢 RAM planning (3-bit, MoE, MLX)

You asked to **assume and decide** realistic ranges. The numbers below are **practical starting points**—verify on your machine.

### Rule-of-thumb components

- **Weights (3-bit):** ≈ `total_params × 0.375 byte` → for **8B params ≈ ~3.0 GB**  
- **Runtime overhead:** MLX graph/tensors/metadata → **~0.6–1.0 GB**  
- **KV-cache:** grows with **context × layers × heads × dtype** → **~0.8–2.5+ GB**

### Indicative peak RAM (batch=1)

| Context window | Estimated peak RAM |
|---|---:|
| **4k tokens**  | **~4.4–5.5 GB** |
| **8k tokens**  | **~5.2–6.6 GB** |
| **16k tokens** | **~6.5–8.8 GB** |

> For ≤2k windows you may see **~4.0–4.8 GB**. Larger windows/batches increase KV-cache and peak RAM.

---

## 🧭 Precision choices for LFM2-8B-A1B (lineup planning)

While this card is **3-bit**, teams often publish multiple precisions. Use this table as a **planning guide** (8B MoE LM; actuals depend on context/batch/prompts):

| Variant | Typical Peak RAM | Relative Speed | Typical Behavior | When to choose |
|---|---:|:---:|---|---|
| **3-bit** *(this repo)* | **~4.4–8.8 GB** | **🔥🔥🔥🔥** | **Direct, concise**, great latency | **Default** on 8–16 GB Macs |
| **6-bit** | ~7.5–12.5 GB | 🔥🔥 | Best quality under quant | Choose if RAM allows |
| **8-bit** | ~9.5–12+ GB | 🔥🔥 | Largest quantized size / highest fidelity | When you prefer simpler 8-bit workflows |

> **MoE caveat:** MoE lowers **compute per token**; unless experts are **paged/partitioned**, **memory** still scales with **total parameters** on a single device.

---

## 🚀 Quickstart (CLI — MLX)

**Deterministic generation**
```bash
python -m mlx_lm.generate \
  --model mlx-community/LFM2-8B-A1B-3bit-MLX \
  --prompt "Summarize the following in 5 concise bullet points:\n<your text>" \
  --max-tokens 256 \
  --temperature 0.0 \
  --device mps \
  --seed 0