EVAL_RESULTS.md

Evaluation Results

Published artifact: FrontAgent Planner 14B MLX full-run adapter.

Training Run

• Upstream base: Qwen/Qwen2.5-Coder-14B-Instruct
• MLX base: mlx-community/Qwen2.5-Coder-14B-Instruct-4bit
• Data: train_v2.json 500 samples, eval_v2.json 100 samples
• MLX split: 500 train, 80 valid, 20 test
• Total training: 1000 iterations (50-iter stable pilot + 950 continuation iterations)
• Learning rate: 1e-5
• LoRA: all layers, rank 32, alpha/scale 64, dropout 0
• Batch size: 1
• Gradient accumulation: 4
• Max sequence length: 2048
• Peak memory observed: 13.249 GB

Metrics

• Iter 1 validation loss after warm start: 0.014
• Iter 100 validation loss: 0.005
• Iter 200 validation loss: 0.001
• Iter 900 validation loss: 0.001
• Final iter 950 validation loss: 0.000
• Final iter 950 train loss: 0.000
• Final test loss: 0.000
• Final test perplexity: 1.000

Notes

The run uses the stable lr=1e-5 setup. A prior lr=1e-4 pilot diverged and was not used for this release. These loss metrics are on the deterministic v2 dataset split; generated-plan quality should still be validated in FrontAgent before production rollout.