README.md

---
library_name: transformers
license: apache-2.0
base_model:
  - Qwen/Qwen3-VL-8B-Instruct
tags:
  - multimodal
  - vision-language
  - tool-use
  - agentic
  - qwen3_vl
  - sft
datasets:
  - Accio-Lab/Metis-ColdStart
language:
  - en
pipeline_tag: image-text-to-text
---

# Metis-8B-ColdStart

**Act Wisely: Cultivating Meta-Cognitive Tool Use in Agentic Multimodal Models**

Metis-8B-ColdStart is the **SFT (Supervised Fine-Tuning) checkpoint** of the Metis framework, fine-tuned from [Qwen3-VL-8B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct) on the curated [Metis-ColdStart](https://huggingface.co/datasets/Accio-Lab/Metis-ColdStart) dataset. This checkpoint serves as the starting point for HDPO reinforcement learning, which produces the final [Metis-8B-RL](https://huggingface.co/Accio-Lab/Metis-8B-RL) model.

[[Paper (arXiv)]](https://arxiv.org/abs/2604.08545) | [[GitHub]](https://github.com/Accio-Lab/Metis) | [[RL Model]](https://huggingface.co/Accio-Lab/Metis-8B-RL) | [[ColdStart Data]](https://huggingface.co/datasets/Accio-Lab/Metis-ColdStart) | [[RL Data]](https://huggingface.co/datasets/Accio-Lab/Metis-RL)

## Model Details

| Attribute | Value |
|---|---|
| Base model | [Qwen3-VL-8B-Instruct](https://huggingface.co/Qwen/Qwen3-VL-8B-Instruct) |
| Training stage | Supervised Fine-Tuning (Cold Start) |
| Training data | [Metis-ColdStart](https://huggingface.co/datasets/Accio-Lab/Metis-ColdStart) (~27K samples) |
| Next stage | → [Metis-8B-RL](https://huggingface.co/Accio-Lab/Metis-8B-RL) (HDPO reinforcement learning) |
| License | Apache-2.0 |

## Cold Start Data Curation Pipeline

The SFT corpus is curated from publicly available tool-augmented multimodal trajectories (DeepEyesV2, V-Interaction, Thyme, OpenMMReasoner) through a rigorous three-stage pipeline:

1. **Eradicating hallucinated environmental dynamics** — Execute all code in a sandbox environment; discard trajectories with execution failures.
2. **Isolating genuine tool necessity** — Filter out samples where the base model achieves pass@8 = 1 without any tools, ensuring only genuinely tool-dependent samples remain.
3. **Multidimensional meta-cognitive filtering** — An LLM judge evaluates visual relevance, reasoning coherence, and tool-use rationale to ensure high quality.

## Training Pipeline

```
Qwen3-VL-8B-Instruct
        │
        ▼  SFT on Metis-ColdStart (~27K samples)
  Metis-8B-ColdStart  ← (this checkpoint)
        │
        ▼  HDPO on Metis-RL (~5K prompts)
   Metis-8B-RL  (final model)
```

## Usage

Please refer to the [GitHub repository](https://github.com/Accio-Lab/Metis) for full installation and inference instructions.

### Installation

```bash
git clone https://github.com/Accio-Lab/Metis.git
cd Metis
pip install -e verl
pip install -e ".[vllm,search_tool,python_code_dep]"
```

## Citation

```bibtex
@article{yan2026metis,
  title={Act Wisely: Cultivating Meta-Cognitive Tool Use in Agentic Multimodal Models},
  author={Yan, Shilin and Tong, Jintao and Xue, Hongwei and Tang, Xiaojun and Wang, Yangyang and Shi, Kunyu and Zhang, Guannan and Li, Ruixuan and Zou, Yixiong},
  journal={arXiv preprint arXiv:2604.08545},
  year={2026}
}
```

## Acknowledgments

Metis is built upon [verl](https://github.com/volcengine/verl), [verl-tool](https://github.com/TIGER-AI-Lab/verl-tool), and [Qwen3-VL](https://github.com/QwenLM/Qwen3-VL).