Kapteeni v1 (kapteeni-v1-intuit) — a Jev-compatible System One decision model

Send a state plus typed questions; get back calibrated probability distributions your code can branch on. No text generation. Kapteeni implements the TypeSafe System One decision-model interface (the wire format of Jev, POST /v1/systemone).

This variant: kapteeni-v1-intuit — sharper decisions. Strongest on well-formed numeric, temporal, and multi-step policy traffic (measured skills-slice accuracy 0.81 vs 0.66 for -meticulous, with better calibration on that slice).

Kapteeni v1 ships as two variants of the same architecture. The other, kapteeni-v1-meticulous, is the conservative-confidence default for unknown or messy traffic.

Quickstart

pip install torch transformers safetensors huggingface_hub   # plus the kapteeni package below
python -m kapteeni.serve --dist . --port 8000        # run from this snapshot
curl localhost:8000/v1/systemone -d '{
  "state": "Help! My payouts have been failing for 3 days.",
  "model": "jev-latest",
  "questions": {
    "is_urgent": {"type": "noul", "instructions": "Does this convey urgency?"},
    "department": {"type": "choice", "instructions": "Which team should handle this?",
      "criteria": {"billing": "Payments, invoicing, refunds",
                    "technical": "Bugs, outages, integrations",
                    "sales": "Pricing, upgrades, new accounts"}},
    "frustration": {"type": "score", "instructions": "How frustrated is the customer?",
      "criteria": ["Calm, just stating facts", "Frustrated but civil", "Very angry"]}
  }}'

Or in-process:

from kapteeni.model import SystemOneModel
m = SystemOneModel.from_dist("<snapshot dir>")
answers, usage = m.evaluate(state, questions)

The kapteeni/ Python package is included in this snapshot; the full repository (data pipeline, training, tests, benchmark history and pre-registrations) lives at https://github.com/TriusAI/Kapteeni (Apache-2.0).

Question types (the Jev contract)

type	answer	readout
noul	{type, noul} — P(yes) in [0,1]	absolute: sigmoid head on the final hidden state; not complement-consistent (P(A) + P(not-A) may differ from 1, matching the reference)
choice	{type, choice, probabilities, confidence}	relative: per-option passes through the backbone, shared head, group softmax; probabilities sum to exactly 1
score	{type, score, legend, probabilities, confidence}	independent levels; score is the probability-weighted expectation and can land between levels

Serving adds a verbalizer blend: the backbone's own next-token yes/no logits are geometrically blended with the trained head (weights in kapteeni-config.json), which measurably improves out-of-domain robustness and calibration. --readout head or --readout verb select pure variants.

Benchmark (JevBench v1.4 public half, self-reported)

231 public decisions, scored with the benchmark's own code, end-to-end through the server. Numbers only — no placement claims; see caveats.

JevBench-style score	Intelligence	top-label ECE	public accuracy
63.18	61.1	0.1196 (Calibration 76.1)	0.714 (easy 1.000 / standard 0.903 / hard 0.468)

Axes common to both variants: Speed 81.0 (p50 0.17 s, p95 1.2 s on an AMD Strix Halo iGPU, x2 self-hosted adjustment) · Cost: 597 input tokens/decision at the hosted list price — verified 2026-09-26: $0.03/M (Novita, qwen3-4b-fp8) = $0.018/1k decisions; nearest official Alibaba tier (qwen-turbo, $0.05/M) = $0.030/1k; an earlier $0.14/M assumption ($0.083/1k) is kept as the pessimistic bound. Under the benchmark's published Jev-class definition (cost <= $0.080/1k decisions, adjusted median latency <= 1.30 s), both variants qualify at verified prices (adjusted p50 0.34 s). Composite = harmonic mean; the Intelligence<50 gate does not apply.

Caveats, stated plainly: self-reported public half (judge and sealed items are private; not an official rank); all serving constants pre-registered on held-out validation (no benchmark selection); Intelligence is renormalized without the sealed judge tier; Calibration shown is the ECE half only. Differences of a few points are within single-seed pipeline noise (~±2-3 composite points, measured).

How it was trained

Qwen3-4B-Instruct-2507 with LoRA r=32 on all attention and MLP projections for one epoch over an 11.1M-token union of the base decision mix (BoolQ/FEVER soft labels, Banking77, CLINC150, GoEmotions, HelpSteer2, synthetic v1) plus 6.3k synth2 items: temporal/numeric with wide format and boundary diversity, multi-hop eligibility/process/fee chains, and grammar-built long policy documents — all ground truth by construction. MNLI OOD gate held 0.88 -> 0.907. Serving constants (temperatures, blend) refit on a deployment-diverse held-out set (mixed-domain val + the synth2 val slice) per a pre-registered protocol; saturation-checked end to end.

Training data provenance

data	license
Qwen3-4B-Instruct-2507 (base)	Apache-2.0
GoEmotions	Apache-2.0
Banking77, HelpSteer2	CC BY 4.0
BoolQ	CC BY 3.0
MultiNLI	CC BY-SA 4.0
FEVER (underlying)	CC BY-SA 3.0
CLINC150	research use (no explicit license)
synthetic items	Apache-2.0 (this project)

Weights: CC BY-SA 4.0 (the ShareAlike terms of MultiNLI/FEVER carry the recommendation; see WEIGHTS-LICENSE.md for the full attribution obligations). Code: Apache-2.0 (LICENSE).

Limitations

• Confidence on messy, adversarial, or out-of-distribution inputs is less reliable than -meticulous (public-half ECE 0.12 vs 0.05): on unknown traffic, prefer -meticulous.
• Probabilities are calibrated in aggregate; individual answers are not guaranteed correct — branch on confidence where it matters.
• Not affiliated with or endorsed by TypeSafe AI; "Jev" is their model and trademark; this is an independent implementation of the documented interface, evaluated on the public JevBench items.