EVLMs-v2 β€” CSA spatial attention + mHC, AdamW (Ablation)

Ablation checkpoint of EVLMs-v2, a medical vision-language model (Swin Transformer vision trunk + Gemma-3-1B language decoder with LoRA, multi-task heads for 14-way CheXpert classification and report generation). This repository isolates CSA spatial attention + mHC, AdamW on the 16,000-sample MIMIC-CXR-VReason subset (777-sample official validation split), batch size 16, bfloat16 autocast, adaptive multi-task loss with per-class positive weighting.

Condition: spatial_attention_type=csa, use_mhc=true, optimizer=AdamW(1e-4) Status: Completed β€” 2 epochs / 2,000 steps (budget-capped ablation) Run: https://wandb.ai/codewithdark-islamia-college-peshawar/evlms-v2/runs/cceknh94

Results

Epoch val loss accuracy
epoch_001 1.0677 0.7019
epoch_002 1.0348 0.7322

Full baseline (mHC + CSA + Muon, same data budget) for reference: val loss 0.8900, accuracy 0.7515, mean AUC 0.6810 at step 6000 (see codewithdark/EVLMs-v2-MIMIC-CXR).

Per-class metrics (last completed epoch)

Class AUC AP
Atelectasis 0.742 0.461
Cardiomegaly 0.717 0.329
Consolidation 0.628 0.541
Edema 0.725 0.553
Enlarged Cardiomediastinum 0.681 0.013
Fracture 0.559 0.087
Lung Lesion 0.522 0.104
Lung Opacity 0.697 0.480
No Finding 0.835 0.929
Pleural Effusion 0.580 0.894
Pleural Other 0.703 0.112
Pneumonia 0.608 0.389
Pneumothorax 0.648 0.873
Support Devices 0.823 0.763

Files

  • final_model/model.safetensors β€” final language-model state dict (where the run completed)
  • best_model.pt β€” best validation checkpoint
  • checkpoints/ β€” periodic step checkpoints
  • artifacts/epoch_XXX/ β€” per-epoch metrics + GradCAM explainability overlays
  • config.json β€” tracking configuration only (secret values redacted)

Intended use & restrictions

Research use only. Trained on MIMIC-CXR-VReason (derived from MIMIC-CXR, PhysioNet) β€” not for clinical decision making. Credential fields in config.json are redacted by design.

Downloads last month
11
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support