LFM3-VL-2B-3D-Coder

Supervised specialisation of LiquidAI/LFM3-VL-2B for generating executable Three.js 3D scenes and GLB-exportable assets from natural-language instructions.

This is an independent, exploratory fine-tune — a quasi-reproduction and extension of ccore/Qwen3.5-4B-3D-Coder, targeting a model half the size (1.92B vs 4.6B parameters) but with higher reported benchmark performance.

What changed relative to ccore's work

	ccore (Qwen3.5-4B-3D-Coder)	This model
Base	Qwen3.5-4B	LFM3-VL-2B
Adaptation	Full fine-tune (+ short LoRA stage)	High-rank QLoRA from the base model (r=64, α=128)
Quantisation	NF4 only for the short LoRA stage	NF4 4-bit QLoRA throughout
Objective	cross-entropy SFT	DFT (Dynamic Fine-Tuning, ICLR 2026) — one-line loss reweighting
Framework	custom full-FT loop	Unsloth + TRL 1.17.1 SFTTrainer
RL	none	none
Data	3,933 scenes	all 3,933 scenes + 1,500 GLB-asset examples

DFT is implemented natively by TRL 1.17.1 via SFTConfig(loss_type="dft"), which computes per_token_loss * log_probs.exp().detach() using a fused/chunked LM head (no full logit materialisation). No RL stage of any kind was used.

Training configuration

• Base: LiquidAI/LFM3-VL-2B (ChatML, <think>…</think> reasoning, 128K context), transformers 5.22.1, trust_remote_code=True, attn_implementation="sdpa".
• QLoRA: bitsandbytes NF4 + double quant, bf16 compute. LoRA r=64, α=128, dropout 0, bias none. Target modules (language model only; vision encoder frozen): q_proj, k_proj, v_proj, o_proj, gate_proj, in_proj, out_proj, w1, w2, w3 — including Liquid AI's per-head attention gate_proj. 65,874,944 trainable params (3.32%).
• Loss: DFT, assistant-only (labels pre-masked; ChatML {% generation %} markers).
• Data: ccore/ThreeJS-3D-Scenes (default, all 3,933 train / 213 validation) plus 1,500 randomly sampled (seed=42) examples from ccore/ThreeJS-3D-GLB-Assets (conversations, 3,501 train / 417 validation). Real validation splits used for eval. max_length=None — no truncation; every example trained complete (max 66,831 tokens).
• Batch: per_device_train_batch_size=1, gradient_accumulation_steps=4 → effective batch size 4. Micro-batch 1 keeps full-length sequences inside 80 GB; effective batch size is unchanged.
• Optimiser: AdamW fused, peak LR 1e-4, cosine decay, 59 warmup steps, weight decay 0.01, β=(0.9, 0.999), ε=1e-8, bf16, 1 epoch (1,359 optimiser steps).
• Memory/compute: Unsloth "unsloth" gradient checkpointing on one NVIDIA A100-SXM4-80GB via Google Colab Pro; ~45 GB peak VRAM at 67K tokens, no CPU offload; ~24 s/optimiser step, ~4,000 tok/s.
• Monitoring / recovery: live status.json + metrics.jsonl pushed every 10 steps, validation loss every 50 steps, generated HTML samples every 200 steps, LoRA checkpoints pushed to this repo every 10 steps, and automatic resume from the latest Hub checkpoint on a fresh runtime.

Repository layout

Path	Contents
merged/	Final LoRA merged into the bf16 base weights (Transformers, trust_remote_code=True)
final_adapter/	Final LoRA adapter
checkpoints/step-*/	Interim LoRA checkpoints every 10 steps
status.json	Live training heartbeat (step, loss, LR, elapsed)
evals/baseline/	Samples from the original, untrained checkpoint
evals/step-*/	Generated Three.js HTML + results at eval steps
gguf/	4-bit Q4_K_M GGUF
training_config.json	Full machine-readable config
TECHNICAL_REPORT.md	Technical report

Usage

Use the same system prompt as the training data:

Use only the Three.js core ES module from
https://cdnjs.cloudflare.com/ajax/libs/three.js/0.160.0/three.module.min.js.
Do not import addons or legacy examples/js paths;
implement any camera controls directly inside the HTML.

For GLB assets, the dataset's second system prompt is used (Godot / Export GLB button).

Recommended sampling — the LFM3-VL-2B model-card defaults, used for all evaluations in this repo:

output_ids = model.generate(
    **inputs, do_sample=True,
    temperature=0.1, top_p=1.0, top_k=50, min_p=0.0,
    repetition_penalty=1.1, max_new_tokens=16384,
)

For comparability with ccore's study, a secondary regime (temperature=0.6, top_p=0.95, top_k=20, repetition_penalty=1.0) is also reported. Allow a large output budget — training examples reach 66K tokens, so in-training evals use max_new_tokens=24576; an 8K cap truncates reasoning before the HTML is emitted. Keep <think> reasoning enabled. During training, evals load the base in NF4 4-bit (GPU is shared); the final evaluation uses the unquantised bf16 merged model.

Limitations

• Exploratory study with synthetic data; evaluation is qualitative (rendered scenes), not a uniform first-attempt benchmark.
• Long generations can loop in <think> before emitting HTML; retries are sometimes needed.
• LFM3-VL-2B is not recommended by its creators for coding; this fine-tune specialises it for one narrow code-generation domain.
• The GGUF requires llama.cpp ≥ b11938 for Lfm3VlForConditionalGeneration support.

Attribution

Base model: Liquid AI, LFM Open License v1.0. Datasets: ccore (MIT / Apache-2.0). DFT: Wu et al., On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectification, ICLR 2026 (arXiv:2508.05629). Framework: Unsloth, TRL, PEFT, bitsandbytes. Technical report: ccore's Qwen3.5-4B-3D-Coder study.