vultr/Ling-3.0-flash-VL-GGUF

Ling 3.0 Flash VL, GGUF

Ling 3.0 Flash VL is a native multimodal Mixture-of-Experts model designed for visual reasoning, agentic workflows, and real-world task execution across images and videos. The architecture combines KDA and Gated MLA. Its ViT encoder and projector integrate visual features for multimodal understanding, reasoning, and action.

This repository holds Vultr's GGUF quantizations of it. They are hybrid quantizations: rather than applying one precision to the whole network, each layer's precision is chosen from a measured importance score, so the bits go where they change the output most.

Model Overview

Attribute	Value
Base model	inclusionAI/Ling-3.0-flash-VL
Architecture	bailingmoe3, Mixture-of-Experts
Parameters	124B total
Active parameters	5.5B per token
Layers	42
Hidden size	2,560
Attention heads	32
Key-value heads	32
Experts	512, 8 activated per token, alongside shared experts
Context length	up to 256K tokens
Vocabulary	157,184
Attention layout	KDA and Gated MLA layers in a 5:1 ratio
License	MIT

Attribute	Value
Base model	inclusionAI/Ling-3.0-flash-VL
Architecture	bailingmoe3, Mixture-of-Experts
Parameters	124B total
Active parameters	5.5B per token
Layers	42
Hidden size	2,560
Attention heads	32
Key-value heads	32
Experts	512, 8 activated per token, alongside shared experts
Context length	up to 256K tokens
Vocabulary	157,184
Attention layout	KDA and Gated MLA layers in a 5:1 ratio
License	MIT

Quantizations

File	Type	Size	KLD vs bf16
Ling-3.0-flash-VL-Q5_K_M.gguf	Q5_K_M	97.15 GB	0.0763
Ling-3.0-flash-VL-Q4_K_M.gguf	Q4_K_M	82.78 GB	0.1391
Ling-3.0-flash-VL-IQ3_M.gguf	IQ3_M	59.74 GB	0.2962
Ling-3.0-flash-VL-IQ3_S.gguf	IQ3_S	59.18 GB	0.3233

File	Type	Size	KLD vs bf16
Ling-3.0-flash-VL-Q5_K_M.gguf	Q5_K_M	97.15 GB	0.0763
Ling-3.0-flash-VL-Q4_K_M.gguf	Q4_K_M	82.78 GB	0.1391
Ling-3.0-flash-VL-IQ3_M.gguf	IQ3_M	59.74 GB	0.2962
Ling-3.0-flash-VL-IQ3_S.gguf	IQ3_S	59.18 GB	0.3233

The vision encoder is in Ling-3.0-flash-VL-mmproj-f16.gguf (875 MB). llama.cpp downloads it automatically with the command under Run, so image input works without extra steps.

KL divergence measures how far a quantized model's next-token probabilities drift from those of the original bf16 model, averaged over every token of the evaluation text. The lower the value, the closer the quantized model is to the original; zero would mean identical output.

Evaluation

Every variant in this repository was evaluated against the original bf16 model for perplexity, KL divergence, top-token agreement, HellaSwag and WinoGrande, under identical conditions. The full results are maintained on the model's page in the Vultr Model Library:

https://docs.vultr.com/models/inclusionai-ling-30/ling-3.0-flash-vl?tab=quantized

That page is the reference for these files. The summary above is an excerpt from it.

Run

llama-server -hf vultr/Ling-3.0-flash-VL-GGUF:Q4_K_M

The model's chat template is embedded in every file, so no extra prompt formatting is needed.

License

These files are derived from inclusionAI/Ling-3.0-flash-VL and are distributed under the same license, MIT. The model is the work of inclusionAI; Vultr produced the quantizations.