Chon-1.0-E4B

Chon-1.0-E4B is a typed-decision model. Given a state (text, JSON or a conversation) and one question with a closed set of options, it returns a calibrated probability for every option:

• choice: pick one of 2-256 labelled options (more than 16 are read in groups, then a final round)
• yes/no: P(yes)
• score: a distribution over ordered levels

It is a LoRA adapter on google/gemma-4-E4B-it (Gemma 4 E4B, about 4B effective parameters) and serves the TypeSafe /v1/systemone wire format used by JevBench.

Serving code, full guide and Docker image: github.com/xoai/chon (MIT). The commands below are enough to run it.

Base model	google/gemma-4-E4B-it, unmodified, loaded alongside this adapter
Adapter	LoRA (~558 MB)
Question types	choice (2-256 options), yes/no, score
Input	state up to 6,000 tokens (longer states are truncated, keeping the start); prompt up to 8,192 tokens
Languages	English; Vietnamese also supported
Hardware	one GPU with 40 GB or more with the vLLM reasoning step, 24 GB without it (measured on one H100 80 GB); on a Mac, Apple silicon with 24 GB or more (--device mps --no-think); also runs on CPU with --no-think
Licence	Apache-2.0, as the base model

Use

pip install "chon[serve] @ git+https://github.com/xoai/chon@v1.1.1"
python -m chon.serve --model ninhnv/chon-1.0-e4b --port 8000      # downloads the base model and this adapter

The first start downloads the base model, google/gemma-4-E4B-it (public, no token needed), into the Hugging Face cache. --device defaults to auto: an NVIDIA GPU if there is one, else Apple's GPU, else the CPU.

On a Mac with Apple silicon and 24 GB or more (uses the Mac's GPU), or on CPU only:

python -m chon.serve --model ninhnv/chon-1.0-e4b --device mps --no-think --port 8000
python -m chon.serve --model ninhnv/chon-1.0-e4b --device cpu --no-think --port 8000

--no-think makes every decision a single forward pass. Use it on a Mac or CPU: the short extra pass for unsure questions generates text, which is slow without a data-centre GPU. Chon-1.0-E2B is the better fit for Macs with less memory.

curl -s localhost:8000/v1/systemone -H 'Content-Type: application/json' -d '{
  "state": "Order #4411 shipped on 3 March; the customer reports it arrived damaged on 9 March.",
  "questions": {"decision": {"type": "choice", "instructions": "Route this ticket.",
    "criteria": {"refund": "Refund or replacement", "shipping": "Delivery status", "other": "Anything else"}}}
}'

The response holds answers.decision with choice and probabilities for choice, noul (P(yes); also as probabilities keyed true / false) for yes/no, or probabilities over the levels for score. usage reports input_tokens and output_tokens: most decisions are a single forward pass with no output tokens, while some take a short extra pass whose generated tokens are counted in output_tokens.

Python

from chon.infer import DecisionModel

dm = DecisionModel.load("ninhnv/chon-1.0-e4b", device="auto")   # NVIDIA GPU, else Apple's GPU, else CPU
answers, usage = dm.predict(
    "Order #4411 shipped on 3 March; the customer reports it arrived damaged on 9 March.",
    {"damaged": {"type": "noul", "instructions": "Did the order arrive damaged?"}})
print(answers["damaged"]["noul"])   # P(yes)

When option labels are only ids ("0", "1", ...), add "option_keys": "hide" to the question so the model sees only the option descriptions.

Faster reasoning (vLLM)

On questions it is unsure about, the model first writes up to 512 tokens of reasoning with the plain base model, then reads its answer. chon.serve hands that writing step to a vLLM server of google/gemma-4-E4B-it (FP8 weights) when one can run: it starts vllm serve itself (from $CHON_VLLM_BIN, vllm on the PATH or /opt/vllm-env/bin/vllm) on a free local port, with the GPU memory left after loading this model, or it uses $CHON_VLLM_URL if you run the server yourself. Without vLLM, or with too little free memory, it uses a static KV cache instead (slower than vLLM, no second copy of the weights). The answer is always read by this model. On a machine that has not run it before, vLLM compiles its kernels at start-up, so the server takes about 5 minutes before /health answers (2 minutes once the compiled kernels are cached; up to 20 on a slow disk).

vLLM goes in its own environment (it brings its own torch):

python -m venv /opt/vllm-env && /opt/vllm-env/bin/pip install uv
/opt/vllm-env/bin/uv pip install --python /opt/vllm-env/bin/python "vllm==0.32.0" --torch-backend=auto

or use the Docker image in the code repository (docker/Dockerfile.serve), which has both environments.

Changes

23 Oct 2026. Same adapter, calibration and thresholds as the first release (revision b7b9ecd); one serving change: the reasoning on unsure questions (about a quarter of questions) is written by vLLM with FP8 weights instead of plain transformers generation. Measured on one H100 80 GB: p95 response 21.4 s -> 3.2 s; held-out capability 77.9 -> 77.7, public 81.8 -> 83.3; official harness hard split 78.4% -> 76.6% (easy and original unchanged). Needs chon 1.1.0 or later for the vLLM path (older versions serve this package with plain generation). On a Mac or CPU nothing changes: keep --no-think.

What is in this repository

File	What it is
adapter_model.safetensors, adapter_config.json	the LoRA adapter (PEFT)
chon_config.json	serving settings (prompt and calibration) read by the chon server
tokenizer files	the base model's tokenizer and chat template, unchanged
README.md	this card

Evaluation (our measurements, not official board numbers)

set	Intelligence	Calibration (proxy)	Capability (proxy)
held-out replica (720)	71.3	84.1	77.7
JevBench public (231)	79.2	87.5	83.3

Serving, measured on one H100 80 GB, one request at a time: median response 0.10 s, p95 3.2 s; 740 input and 121 output tokens per decision on average.

On CPU only (8 threads of an Intel Xeon Platinum 8480+, bf16, one request at a time, --no-think): median response 3.0 s over 30 short held-out questions (about 230 input tokens; typical questions are about three times longer and take roughly proportionally longer). On a MacBook Pro with 18 GB (Apple M3 Pro) the model did not fit in memory alongside the system; use a Mac with 24 GB or more.

Official JevBench harness (typesafe adapter, public files) against the served package, accuracy: easy 100.0%, original 98.6%, hard 76.6%.

• Held-out set: 720 items in the JevBench v1.5 sealed-set mix (Choice 360 / yes/no 180 / score 180; tiers 14/28/28/30%), written for this project and never trained on.
• Scoring: the board's published formulas where they are public (equal type weights, tier weights 10/20/30/40, yes/no answers inside 0.2-0.8 count as wrong). The board's calibration mapping is not fully published, so the calibration column is our proxy.

Licence

Apache-2.0, as google/gemma-4-E4B-it.