Instructions to use frontier-infra/jebadiah-4b-v2-MLX-8bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use frontier-infra/jebadiah-4b-v2-MLX-8bit with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("frontier-infra/jebadiah-4b-v2-MLX-8bit")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use frontier-infra/jebadiah-4b-v2-MLX-8bit with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "frontier-infra/jebadiah-4b-v2-MLX-8bit"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "frontier-infra/jebadiah-4b-v2-MLX-8bit"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

MLX LM

How to use frontier-infra/jebadiah-4b-v2-MLX-8bit with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "frontier-infra/jebadiah-4b-v2-MLX-8bit"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "frontier-infra/jebadiah-4b-v2-MLX-8bit"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "frontier-infra/jebadiah-4b-v2-MLX-8bit",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Hermes Agent

How to use frontier-infra/jebadiah-4b-v2-MLX-8bit with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "frontier-infra/jebadiah-4b-v2-MLX-8bit"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default frontier-infra/jebadiah-4b-v2-MLX-8bit

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use frontier-infra/jebadiah-4b-v2-MLX-8bit with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "frontier-infra/jebadiah-4b-v2-MLX-8bit"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "frontier-infra/jebadiah-4b-v2-MLX-8bit" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Just want to run it? Download jebadiah-9b-v2-1-GGUF and open it in llama.cpp, Ollama or LM Studio.

Which one should I use? Start with the 9B. It's smaller and faster, and it's what most people run. Pick the 27B if you have the memory and want the higher score (Decision Index Full 55.94 against the 9B's 44.91, board data of 2026-10-15). On Apple silicon, use the MLX builds of the same models. You can try the 4B in the browser first: jebadiah-playground.

This repo: the 4B v2 MLX build, 8-bit, for Apple silicon with mlx-lm.

This is an earlier version. For new work, use v2.1 above.

Jebadiah 4B v2 MLX 8bit

An 8bit MLX build of jebadiah-4b-v2 for Apple silicon, with all model files at the repository root (group size 64). Jebadiah is a decision model, not a chat model. It answers a typed question (choice, noul or score) with a probability for every option, read from one forward pass. Nothing is generated. Code, trainer and evals: getainode/jebadiah.

Results and docs

Project site, with every result and how to run the models: jebadiah.ai.
For the family: Decision Index 0.2.1: the 27B scores 54.67, #5 of 67 open models (as of 2026-09-26). This is the board's own number; the maintainer validated my run and put it on the leaderboard. Run record.
JevBench v1.4.2: on its 231 public items, run through its own harness, Jebadiah 27B scores 0.866, the same as Jev 1.13.0; Jebadiah 9B v2 scores 0.818. This is my own run on the public items, not the official board, which also uses sealed items. Details and caveats. The 4B has not been run on either.
All sizes: the Hugging Face collection, mirrored on ModelScope.

Which one should I use? For local use, start with Jebadiah 9B v2 GGUF. On Apple silicon, use an MLX build: 27B, 9B v2 or 4B v2. For vLLM or fine-tuning, use the full weights: 27B, 9B v2 or 4B v2. Setup for every runtime, and for JDE: Run Jeb locally.

Files

Every file was checked on the 260 held-out questions the merged weights were checked on, and compared with the merged bf16 weights and with the training run's own eval records.

File	Size	Same answer as bf16	Same as the run	choice + noul	score	Prob. diff median / max
`8bit` (root)	4.5 GB	252 / 260	253 / 260	169 / 173	84 / 87	0.004 / 0.110
bf16 weights			259 / 260	172 / 173	87 / 87	0.002 / 0.019

This 8bit build needs about its own size in unified memory, plus about 1 GB for a 2k-token prompt.

8bit changes 8 of 260 answers against bf16 (3 on score questions) and moves probabilities more (median 0.004, max 0.11). Use it only when a larger build does not fit.

Run it

The script renders the prompt exactly as AINode does, runs one forward pass with mlx-lm, multiplies the last hidden state by the option labels' output-head rows in fp32 and applies temperatures.json (choice 1.1167, noul 1.3319, score 0.8312). Needs mlx-lm 0.31 or newer (qwen3_5 support).

pip install "mlx-lm>=0.31"
hf download frontier-infra/jebadiah-4b-v2-MLX-8bit --local-dir jebadiah-4b-v2-MLX-8bit
cd jebadiah-4b-v2-MLX-8bit
python scripts/decide_mlx.py --model . --request scripts/example-request.json

--no-temperatures returns the raw probabilities.

Use the local root folder with --model . as shown above. The copied script's Hub-ID mode expects the legacy precision subfolders; download this repo first.

HelpSteer2-like traffic. A held-out HelpSteer2 check (2026-09-29): on the 418 nvidia/HelpSteer2 validation rows that no reported evaluation set uses (natural label mix, never in the training pool), this model's score answers want a temperature of 1.26, not 0.83, and still 1.08 after reweighting to a flat label mix. If your traffic looks like HelpSteer2, keep the older score temperature, 1.20 (the train fit in temperatures.json; every script here takes --temperatures with a file of your own), or refit on your own labels. v3's calibration split will draw HelpSteer2 from held-out data.

On example-request.json (this 8bit build):

{
 "route": {"type": "choice", "choice": "billing", "confidence": 0.394292, "probabilities": {"billing": 0.596195, "support": 0.0839, "sales": 0.319905}},
 "urgent": {"type": "noul", "noul": 0.147991}
}

How it was measured

Jevals PubMedQA, Banking77 (77 options) and HelpSteer2, plus Nimble: the merge check's fixed sample (seed 20260925), the run's option order and the run's temperatures, so the probability differences compare like with like (the shipped temperatures.json since 2026-09-29 uses 0.8312 for score; top picks do not depend on it). "Same answer" is the top option; "prob. diff" is the largest change on any option against the run's CUDA record. Records: eval/agreement-8bit.json.