jsbeaudry/Fatiman-TTS-v1-onnx

Pocket TTS Kreyòl — ONNX for phones

Try it in your browser: The Trio Serie Space runs Fatiman with the other two models of the series: the Fatiman tab reads your text with the nine voices, running these ONNX files on CPU, and the Klara tab speaks Makandal's answers. More on the Fatiman page of thetrio.space.

This is the Haitian Creole fine-tune of Pocket TTS (from french_24l), exported to ONNX for on-device use with ONNX Runtime. It is what the Klara mobile app (Expo / React Native, Android) loads.

Files

file	what	size
text_conditioner_int8.onnx	token ids → text embeddings	16 MB
flow_lm_main_kv_q4.onnx	24-layer backbone LM, append-only KV, 4-bit weights (MatMulNBits, block 32)	190 MB
flow_lm_flow_int8.onnx	flow head: conditioning + noise → 32-dim latent (one LSD step)	10 MB
mimi_decoder_int8.onnx	Mimi decoder: latents → 24 kHz audio, streaming state	23 MB
alt/flow_lm_main_kv_int8.onnx	the same LM with INT8 weights (dynamic quant), slightly better, 1.6× larger	304 MB
bundle.json	dims, state manifests, kv layout, lm_file, Mimi pass-through outputs
tokenizer.json	the fine-tune's Unigram tokenizer (not the French sentencepiece)
bos_before_voice.f32	BOS embedding inserted before the voice prompt (float32, 1024)
voices/*.f32, voices/voices.json	voice prompt embeddings (float32 [1, frames, 1024]), computed in PyTorch
export/	scripts that produced the LM files

The LM interface (append-only KV)

The stock export passes a fixed-size KV cache (100 MB) in and out of the graph every frame. On a phone, that copy cost more than the model itself. This export takes each layer's past keys and values at their used length, and returns only the new positions:

• inputs: sequence [1, S, 32] (NaN = BOS), text_embeddings [1, X, 1024], past_k_i / past_v_i [1, P, 16, 64] for i = 0..23 (P may be 0)
• outputs: conditioning [1, 1024], eos_logit [1, 1], new_k_i / new_v_i [1, S+X, 16, 64]

The caller appends new_* to its own buffers. The Klara app uses one time-major buffer per layer and feeds zero-copy prefix views.

1. Voice: run once with text_embeddings = [bos_before_voice, voice frames] and empty past. Keep the outputs as that voice's prefix.
2. Each text chunk: reset the cache to the voice prefix, then run the text embeddings.
3. Each frame: run sequence = the previous latent (NaN for the first frame) → flow head (s=0, t=1, x ~ N(0, 0.7), x += flow) → latent.
4. Stop a few frames after eos_logit > -4.

export/export_flow_lm_kv.py checks the export against the stock streaming path: an exact match in PyTorch, and within 2e-6 for the ONNX graph.

Quality and speed

Qwen3-ASR Kreyòl CER on 20 held-out sentences, man3 voice, averaged over 2 sampling seeds:

LM	CER
stock INT8 export (reference)	10.2% (one seed)
alt/flow_lm_main_kv_int8.onnx	9.8%
flow_lm_main_kv_q4.onnx	10.7% (within seed-to-seed noise)

Speed on a Galaxy A16 (Exynos 1330, ORT 2 threads):

• LM frame: 43 ms alone; the stock export took 311 ms.
• Full loop RTF: 1.3–1.45, thermally capped.
• First sound: 1.2–2 s.

Notes

• Public since 2026-10-04 so the Klara app can download the files on first launch.