library_name: custom
tags:
- vision
- ripeness
- tomato
- jepa
- preview
plantjepa-preview
Tomato ripeness from a single fruit crop. Give it a photo plus a bounding box, get back the ripeness stage with a confidence. Small JEPA encoder plus linear head trained on pooled multi-farm boxes (BUTom-ST21, strawberry, Laboro-train) — roughly 10x smaller than SigLIP2 with far better cross-farm generality.
Use in 30 seconds
pip install -r requirements.txt
from huggingface_hub import snapshot_download
from inference import RipenessModel
folder = snapshot_download("DavidSeyserHF/sprout-preview")
model = RipenessModel.load(f"{folder}/plantjepa_preview.pt", f"{folder}/calibration.json")
print(model.predict("tomato.jpg", bbox=(x1, y1, x2, y2)))
Or from the command line:
python inference.py --checkpoint plantjepa_preview.pt --calibration calibration.json \
--image tomato.jpg --bbox X1 Y1 X2 Y2
bbox is pixels in the original image: (xmin, ymin, xmax, ymax). The model
adds a little context around the box itself, so a tight box is fine.
What you get back
Example output (stage and confidence observed; remaining values illustrative):
{
"predicted_stage": "mature_unripe",
"confidence": 0.5373,
"stage_probabilities": {
"immature": 0.01,
"mature_unripe": 0.54,
"ripening": 0.30,
"ripe": 0.14,
"overripe": 0.01
},
"ordinal_maturity_index_0_100": 38.2,
"temperature": 0.8586
}
Stages are immature, mature_unripe, ripening, ripe, overripe. The maturity
index is an expected rank over these stages, not a physical measurement.
Confidences are temperature-scaled (see calibration.json); predictions are
unchanged by it. Rule of thumb: route anything under ~0.8 confidence to human
review and spot-check the rest.
How good is it?
Home rows 5–6 (5,556 crops) vs Laboro test, foreign farm (1,996 boxes):
| Model | Home acc / F1 | Away acc / F1 |
|---|---|---|
| JEPA multi-source (this release) | 86.84% / 77.81% | 66.1% / 0.414 |
| JEPA single-farm previous | 88.01% / 79.46% | 48.9% / 0.351 |
| SigLIP2 frozen reference (~10x larger) | 91.45% / 79.92% | — |
Selective prediction travels even where absolute confidence does not: on Laboro, accuracy rises 0.66 → 0.89 from full coverage to the 0.95 threshold.
Limitations
- Home evidence is 2 independent greenhouse rows: point estimates, no reliable 95% CI, no equivalence claim vs SigLIP2.
- BUTom train labels are pseudo-labels; all evaluations are hand-corrected.
- The temperature in
calibration.jsonwas fit at home and is valid there only; abroad it hurts (away ECE 8.4 → 13.0). Re-fit thresholds — and ideally the temperature — per deployment instead of trusting these values. - Calibration fixes average confidence, not worst-case confident mistakes.