Blame for README.md

---
base_model:
- meituan-longcat/LongCat-Flash-Lite
license: mit
pipeline_tag: text-generation
tags:
- engramedit
- knowledge-editing
- conditional-memory
---

# EngramEdit: ZsRE · 2K

## Overview

Conditional memory expands an LLM's capacity through learned **n-gram embeddings** that participate in its computation. **EngramEdit** enables decoupled knowledge updates by updating these embeddings while keeping the Transformer backbone fixed. Revised facts remain usable across different expressions and in multi-hop reasoning, while unrelated knowledge and general capabilities are largely preserved.

This repository provides the EngramEdit checkpoint for **2,000 ZsRE edits** on [LongCat-Flash-Lite](https://huggingface.co/meituan-longcat/LongCat-Flash-Lite). The checkpoint contains the learned conditional memory updates. Load it together with the base model using the EngramEdit code below.

## Links

[📄 Paper](https://arxiv.org/abs/2610.10533) · [🤗 Hugging Face Paper](https://huggingface.co/papers/2610.10533) · [🌐 Project Page](https://modalitydance.github.io/EngramEdit/) · [💻 GitHub](https://github.com/ModalityDance/EngramEdit) · [🧪 Usage Example](https://github.com/ModalityDance/EngramEdit#usage-example) · [📊 Evaluation](https://github.com/ModalityDance/EngramEdit#direct-evaluation)

**Checkpoints:** [🤗 CounterFact · 2K](https://huggingface.co/ModalityDance/EngramEdit-LongCat-Flash-Lite-MCF-2K) · [🤗 ZsRE · 2K](https://huggingface.co/ModalityDance/EngramEdit-LongCat-Flash-Lite-ZsRE-2K) · [🤗 MQuAKE · 3K](https://huggingface.co/ModalityDance/EngramEdit-LongCat-Flash-Lite-MQuAKE-3K)

## Usage

Clone the [code repository](https://github.com/ModalityDance/EngramEdit), follow its [installation instructions](https://github.com/ModalityDance/EngramEdit#1-installation), and run the example from that checkout. Download the base model once:

```bash
hf download meituan-longcat/LongCat-Flash-Lite --local-dir data/LongCat-Flash-Lite
```

The example compares the model's response before and after loading the checkpoint. No editing, expression generation, or frequency-cache preparation is needed.

This example uses a ZsRE question and its target answer. The code prints the actual responses before and after loading the checkpoint.

```python
from pathlib import Path

import torch
from huggingface_hub import hf_hub_download

from experiments.utils import load_model
from EngramEdit import prepare_engramedit_model

# Load the base model.
model, tokenizer = load_model("data/LongCat-Flash-Lite", "bfloat16")
prompt = "What university did Watts Humphrey attend?"
target = "Illinois Institute of Technology"


@torch.inference_mode()
def answer(prompt):
    inputs = tokenizer(prompt, return_tensors="pt").to(
        model.get_input_embeddings().weight.device
    )
    output = model.generate(
        **inputs, do_sample=False, max_new_tokens=32,
        pad_token_id=tokenizer.pad_token_id,
    )
    return tokenizer.decode(
        output[0, inputs.input_ids.shape[1]:], skip_special_tokens=True
    )


# Inspect the query and the target fact.
print("Prompt:", prompt)
# Prompt: What university did Watts Humphrey attend?
print("Target answer:", target)
# Target answer: Illinois Institute of Technology

print("Before loading:", answer(prompt))

# Download and load the learned conditional memory updates.
state_path = hf_hub_download(
    repo_id="ModalityDance/EngramEdit-LongCat-Flash-Lite-ZsRE-2K",
    filename="engramedit_state.pt",
    local_dir="checkpoints/zsre",
)
model = prepare_engramedit_model(model, state_dir=Path(state_path).parent)

# Ask the same question with the checkpoint loaded.
print("After loading:", answer(prompt))
```

## Citation

```bibtex
@misc{cai2026engramedit,
  title  = {EngramEdit: Decoupled Knowledge Updates in LLMs through Conditional Memory},
  author = {Hongru Cai and Ran Wei and Wenjie Wang and Chengfa Wu and Ning Song and Yongqi Li and Wenjie Li},
  year   = {2026},
  eprint = {2610.10533},
  archivePrefix = {arXiv},
  primaryClass = {cs.CL},
  url    = {https://arxiv.org/abs/2610.10533}
}
```

## License

This checkpoint is released under the [MIT License](https://github.com/ModalityDance/EngramEdit/blob/main/LICENSE). The base model is distributed separately under its [own license](https://huggingface.co/meituan-longcat/LongCat-Flash-Lite/blob/main/LICENSE).