itsmechico/moss-tts-nano-coreml-ane

MOSS-TTS-Nano 100M for Core ML on the Neural Engine

A Core ML re-export of MOSS-TTS-Nano-100M and the decoder of MOSS-Audio-Tokenizer-Nano (both Apache-2.0, by the OpenMOSS team) that runs with MLComputeUnits.cpuAndNeuralEngine, without the GPU, so it keeps working in an iOS app playing in the background.

Sources: model 44502f80dbf9743528fa921cc544d662c685ebec, codec 6aa02b01e445cc585582cf0ba480bc3ea6c8dd68.

Differences from the upstream graphs

• No MLState. Every graph is static and stateless; the host owns the KV caches and passes them as inputs. Prompt lengths and contexts are buckets.
• All fp16. RoPE angles come from the host in float64, so phases do not accumulate fp16 error.
• The local decoder's 17 steps per frame are one call, with top-k, top-p, temperature and repetition-penalty sampling in the graph (Gumbel-max with noise from the host).
• The codec decodes 12 frames per call with its sliding-window caches as inputs and outputs.

Files

File	Functions	Size
GlobalDecoder	prefill_{128,256,512}, step_{256,512,1024}	212 MB
LocalDecoder	frame_16, frame_12	61 MB
CodecDecoder	chunk_12	25 MB

Each model is present as .mlpackage and as .mlmodelc (compiled for iOS 26). Load each function with MLModelConfiguration.functionName. config.json holds the buckets, token ids, sampling defaults and quality levels; tokenizer.model / tokenizer.json are the upstream SentencePiece tokenizer.

Measurements (Apple M5 Max, macOS 27, .cpuAndNeuralEngine)

Stage	Neural Engine share of cost	Time per call	Parity against upstream fp32
prefill_256	0.86	7.8 ms	last hidden state cosine 0.99998
step_256 / step_512	0.71 (the rest is the embedding gather)	2.0 / 2.5 ms	min cosine 0.99992 over 60 frames
frame_16	1.0	2.4 ms	sampler exact within 2e-7
chunk_12	1.0	5.4 ms (0.96 s of audio)	34.8 dB SNR over 206 frames

Real-time factor with the full loop in Swift: 0.069 (Portuguese) and 0.066 (English) with 16 codebooks.

Whisper large-v3-turbo word error rate on a test sentence:

• English: 0.00 for both PyTorch and Core ML.
• Portuguese: 0.25 for PyTorch and 0.23 for Core ML, averaged over 6 seeds.

The Portuguese output tends toward a European accent even with Brazilian voice prompts. The upstream model does the same.

Voices

A voice is a prompt of codec codes, in voices/<name>.json, next to the clip it was encoded from.

Voice	Language	Source	License
rachel	pt-BR	LibriVox, Dom Casmurro section 2, read by Rachel Moraes	Public Domain Mark 1.0
thiago	pt-BR	LibriVox, Dom Casmurro section 13, read by Thiago	Public Domain Mark 1.0
ava	en	upstream assets/audio/en_2.wav	Apache-2.0
adam	en	upstream assets/audio/en_4.wav	Apache-2.0

Generation loop

1. Build the prompt rows (17 int32 each: a text id and 16 codes) from config.json's prompt strings, the voice codes and the sentence's SentencePiece ids.
2. Run prefill_P for the smallest bucket that fits, then for each frame:
   • call frame_D to sample the frame's codes and end flag;
   • call step_C with the new row to get the next hidden state and the new key and value;
   • every 12 frames, call chunk_12 to decode audio.
3. Output is 48 kHz stereo, 12.5 frames per second.

The quality levels in config.json combine codebook depth (12 or 16) with a retry budget that redraws takes whose length falls outside 1.8–4.2 frames per text token.

License

Apache-2.0, as the upstream model and codec. The LibriVox clips are in the public domain.