---
library_name: custom
tags: [vision, ripeness, tomato, jepa, preview]
---

# plantjepa-preview

Tomato ripeness from a single fruit crop. Give it a photo plus a bounding box,
get back the ripeness stage with a confidence. Small JEPA encoder plus linear
head trained on pooled multi-farm boxes (BUTom-ST21, strawberry, Laboro-train)
— roughly 10x smaller than SigLIP2 with far better cross-farm generality.

## Use in 30 seconds

```bash
pip install -r requirements.txt
```

```python
from huggingface_hub import snapshot_download
from inference import RipenessModel

folder = snapshot_download("DavidSeyserHF/sprout-preview")
model = RipenessModel.load(f"{folder}/plantjepa_preview.pt", f"{folder}/calibration.json")
print(model.predict("tomato.jpg", bbox=(x1, y1, x2, y2)))
```

Or from the command line:

```bash
python inference.py --checkpoint plantjepa_preview.pt --calibration calibration.json \
  --image tomato.jpg --bbox X1 Y1 X2 Y2
```

`bbox` is pixels in the original image: `(xmin, ymin, xmax, ymax)`. The model
adds a little context around the box itself, so a tight box is fine.

## What you get back

Example output (stage and confidence observed; remaining values illustrative):

```json
{
  "predicted_stage": "mature_unripe",
  "confidence": 0.5373,
  "stage_probabilities": {
    "immature": 0.01,
    "mature_unripe": 0.54,
    "ripening": 0.30,
    "ripe": 0.14,
    "overripe": 0.01
  },
  "ordinal_maturity_index_0_100": 38.2,
  "temperature": 0.8586
}
```

Stages are `immature, mature_unripe, ripening, ripe, overripe`. The maturity
index is an expected rank over these stages, not a physical measurement.
Confidences are temperature-scaled (see `calibration.json`); predictions are
unchanged by it. Rule of thumb: route anything under ~0.8 confidence to human
review and spot-check the rest.

## How good is it?

Home rows 5–6 (5,556 crops) vs Laboro test, foreign farm (1,996 boxes):

| Model | Home acc / F1 | Away acc / F1 |
|---|---|---|
| JEPA multi-source (this release) | 86.84% / 77.81% | 66.1% / 0.414 |
| JEPA single-farm previous | 88.01% / 79.46% | 48.9% / 0.351 |
| SigLIP2 frozen reference (~10x larger) | 91.45% / 79.92% | — |

Selective prediction travels even where absolute confidence does not: on
Laboro, accuracy rises 0.66 → 0.89 from full coverage to the 0.95 threshold.

## Limitations

- Home evidence is 2 independent greenhouse rows: point estimates, no
  reliable 95% CI, no equivalence claim vs SigLIP2.
- BUTom train labels are pseudo-labels; all evaluations are hand-corrected.
- The temperature in `calibration.json` was fit at home and is valid there
  only; abroad it hurts (away ECE 8.4 → 13.0). Re-fit thresholds — and
  ideally the temperature — per deployment instead of trusting these values.
- Calibration fixes average confidence, not worst-case confident mistakes.