Mitchins/led-narrative-factual-16k

LED Narrative Factual 16k

Warm-up checkpoint for factual chapter summaries, built to ground longer outline models.

• Base model: allenai/led-base-16384
• Warm start: paragraph_baseline_models/led_base_baseline/checkpoint-6500
• Dataset: gemini-chapter-summary.jsonl (~38k chapters, avg ~950 tokens)
• Tokenization: LED tokenizer, max_source_length=16384, max_target_length=128
• Training: batch 8 (per device) × grad accum 4 (effective 32), lr 2e-5, 3 epochs, linear warmup 1k steps
• Validation metrics (400 chapters, beam=4, length_penalty=0.6):
  • ROUGE-1: 0.3523
  • ROUGE-2: 0.1045
  • ROUGE-L: 0.2490
  • METEOR: 0.2372
  • Avg generation length: 34.7 words

Use this checkpoint as the factual summarization baseline before training creative long-form outline models.