owensong commited on
Commit
9f6e0ef
Β·
verified Β·
1 Parent(s): 9528ca8

Document managed four-thread CPU runtime

Browse files
Files changed (3) hide show
  1. README.md +41 -10
  2. docs/EVALUATION.md +38 -18
  3. release_manifest.json +4 -4
README.md CHANGED
@@ -70,9 +70,9 @@ These are held-out text generations, not reconstructions of training audio. Each
70
 
71
  No single metric captures TTS quality. Inflect v2 reports **human preference**, **predicted naturalness**, **multi-ASR intelligibility**, **complete footprint**, and **runtime** separately rather than compressing them into one unverifiable score.
72
 
73
- | Community preference ↑ | UTMOS22 ↑ | Two-ASR semantic WER ↓ | Complete FP32 weights ↓ |
74
- | ---: | ---: | ---: | ---: |
75
- | **66.2%** | **4.395** | **3.99%** | **37.53 MB** |
76
 
77
  The headline row always refers to **Inflect-Micro-v2**. Detailed competitor results and protocol boundaries are kept visible below.
78
 
@@ -131,14 +131,45 @@ transcription errors.
131
 
132
  </details>
133
 
134
- ### 4. Runtime status
135
 
136
- The release supports CPU and CUDA inference through the same public API. A
137
- previous local CPU table has been withdrawn because background host saturation
138
- made its cross-system throughput figures non-reproducible. A clean-host rerun
139
- will be published with the complete environment, thread policy, warmup count,
140
- matched prompts, and raw per-utterance timings. Until then, this card makes no
141
- headline CPU-speed claim.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
142
 
143
  ### 5. Complete weight footprint
144
 
 
70
 
71
  No single metric captures TTS quality. Inflect v2 reports **human preference**, **predicted naturalness**, **multi-ASR intelligibility**, **complete footprint**, and **runtime** separately rather than compressing them into one unverifiable score.
72
 
73
+ | Community preference ↑ | UTMOS22 ↑ | Two-ASR semantic WER ↓ | Complete FP32 weights ↓ | 4-thread CPU throughput ↑ |
74
+ | ---: | ---: | ---: | ---: | ---: |
75
+ | **66.2%** | **4.395** | **3.99%** | **37.53 MB** | **6.28Γ— real-time** |
76
 
77
  The headline row always refers to **Inflect-Micro-v2**. Detailed competitor results and protocol boundaries are kept visible below.
78
 
 
131
 
132
  </details>
133
 
134
+ ### 4. CPU runtime
135
 
136
+ Both Inflect releases synthesize comfortably faster than real time on CPU. The
137
+ managed reference run used a Hugging Face **CPU Upgrade** instance (8 vCPU,
138
+ 32 GB RAM) with **four framework threads**, end-to-end text-to-waveform timing,
139
+ and 100 fixed Modern400 prompts. Three complete passes were recorded; the first
140
+ cache-building pass was excluded and the table pools passes two and three.
141
+
142
+ | Release | Steady-state RTF ↓ | Audio / wall time ↑ |
143
+ | --- | ---: | ---: |
144
+ | **Inflect-Micro-v2** | **0.1593** | **6.28Γ—** |
145
+ | **Inflect-Nano-v2** | **0.0933** | **10.72Γ—** |
146
+
147
+ These are package-level results from the public PyTorch runtime, not a claim
148
+ that Inflect is the fastest compact TTS system. Hardware, frontend behavior,
149
+ framework, compilation, and thread policy all affect small-model measurements.
150
+
151
+ <details>
152
+ <summary><strong>Open directional compact-system speed context</strong></summary>
153
+
154
+ The same managed CPU and four-thread policy were used for a shorter comparator
155
+ pass: the identical 50-prompt prefix, repeated twice. KittenTTS and Piper are
156
+ equal-work pooled across their two tested voices.
157
+
158
+ | System | Audio / wall time ↑ |
159
+ | --- | ---: |
160
+ | Piper Low | 31.37Γ— |
161
+ | KittenTTS Nano | 13.33Γ— |
162
+ | **Inflect-Nano-v2** | **10.72Γ—** |
163
+ | Supertonic 3 Β· 3-step | 10.15Γ— |
164
+ | **Inflect-Micro-v2** | **6.28Γ—** |
165
+ | Supertonic 3 Β· 8-step | 4.37Γ— |
166
+
167
+ Because Inflect uses the larger 100-prompt steady-state run while comparator
168
+ rows use the shorter 50-prompt confirmation pass, this table is deployment
169
+ context rather than a perfectly matched speed leaderboard. Several comparators
170
+ also use optimized ONNX runtimes while this Inflect release uses PyTorch.
171
+
172
+ </details>
173
 
174
  ### 5. Complete weight footprint
175
 
docs/EVALUATION.md CHANGED
@@ -14,10 +14,10 @@ behind those results.
14
 
15
  ## Evidence at a glance
16
 
17
- | Release | Complete parameters | FP32 weights | UTMOS22 ↑ | Two-ASR semantic WER ↓ | Community preference ↑ |
18
- | --- | ---: | ---: | ---: | ---: | ---: |
19
- | **Inflect-Micro-v2** | **9,356,513** | **37.53 MB** | **4.395** | **3.99%** | **66.2%** |
20
- | **Inflect-Nano-v2** | **3,966,721** | **15.97 MB** | **4.386** | **4.21%** | **63.9%** |
21
 
22
  ![Community blind listening](../assets/evidence/human-preference.svg)
23
 
@@ -161,22 +161,42 @@ balanced tournament or formal MOS.
161
 
162
  ## Runtime and throughput
163
 
164
- ### Current publication status
165
 
166
- The previous Ryzen 9 3900X comparison is withdrawn. The host was under sustained
167
- background saturation and the harness did not enforce the same thread policy
168
- inside every PyTorch and ONNX runtime. Those numbers are not release evidence.
 
 
169
 
170
- A replacement CPU benchmark must pass all of these gates before publication:
171
-
172
- 1. named, idle host with sustained preflight load below the declared threshold;
173
- 2. one isolated process per system;
174
- 3. identical frozen prompt order;
175
- 4. fixed warmup count and fixed thread budget;
176
- 5. explicit PyTorch and ONNX intra/inter-op thread configuration;
177
- 6. end-to-end timing including frontend and waveform generation;
178
- 7. raw per-utterance latency, audio duration, RTF, provider, memory, and host load;
179
- 8. repeated runs with median and dispersion, not one lucky pass.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
180
 
181
  The public runtime also collapses training-time weight normalization once,
182
  immediately after loading the checkpoint. A fixed-seed regression test produced
 
14
 
15
  ## Evidence at a glance
16
 
17
+ | Release | Complete parameters | FP32 weights | UTMOS22 ↑ | Two-ASR semantic WER ↓ | Community preference ↑ | 4-thread CPU ↑ |
18
+ | --- | ---: | ---: | ---: | ---: | ---: | ---: |
19
+ | **Inflect-Micro-v2** | **9,356,513** | **37.53 MB** | **4.395** | **3.99%** | **66.2%** | **6.28Γ—** |
20
+ | **Inflect-Nano-v2** | **3,966,721** | **15.97 MB** | **4.386** | **4.21%** | **63.9%** | **10.72Γ—** |
21
 
22
  ![Community blind listening](../assets/evidence/human-preference.svg)
23
 
 
161
 
162
  ## Runtime and throughput
163
 
164
+ ### Managed CPU reference
165
 
166
+ The published CPU reference uses a Hugging Face **CPU Upgrade** instance
167
+ (8 vCPU, 32 GB RAM) with four framework threads and end-to-end
168
+ text-to-waveform timing. Inflect received 100 fixed Modern400 prompts across
169
+ three complete passes. The first pass populated runtime caches; the published
170
+ steady-state result pools passes two and three.
171
 
172
+ | Release | Steady-state RTF ↓ | Audio / wall time ↑ |
173
+ | --- | ---: | ---: |
174
+ | **Inflect-Micro-v2** | **0.1593** | **6.28Γ—** |
175
+ | **Inflect-Nano-v2** | **0.0933** | **10.72Γ—** |
176
+
177
+ The earlier Ryzen 9 3900X CPU comparison is superseded because background host
178
+ saturation and inconsistent framework thread policies made it unsuitable as
179
+ release evidence.
180
+
181
+ ### Directional compact-system context
182
+
183
+ The same managed CPU and four-thread policy were used for a shorter comparator
184
+ confirmation: the identical 50-prompt prefix repeated twice. KittenTTS and
185
+ Piper are equal-work pooled across two voices.
186
+
187
+ | System | Audio / wall time ↑ |
188
+ | --- | ---: |
189
+ | Piper Low | 31.37Γ— |
190
+ | KittenTTS Nano | 13.33Γ— |
191
+ | **Inflect-Nano-v2** | **10.72Γ—** |
192
+ | Supertonic 3 Β· 3-step | 10.15Γ— |
193
+ | **Inflect-Micro-v2** | **6.28Γ—** |
194
+ | Supertonic 3 Β· 8-step | 4.37Γ— |
195
+
196
+ This is deployment context, not a perfectly matched speed leaderboard:
197
+ Inflect's figures come from the larger steady-state run, comparator rows use the
198
+ shorter confirmation pass, and several comparator packages use optimized ONNX
199
+ while the public Inflect package uses PyTorch.
200
 
201
  The public runtime also collapses training-time weight normalization once,
202
  immediately after loading the checkpoint. A fixed-seed regression test produced
release_manifest.json CHANGED
@@ -89,8 +89,8 @@
89
  },
90
  {
91
  "path": "docs/EVALUATION.md",
92
- "bytes": 11854,
93
- "sha256": "b2aaf6f467b60d7414f4a4303e09213c382c3e73063deddb76268068dae62730"
94
  },
95
  {
96
  "path": "docs/EXPORTS.md",
@@ -294,8 +294,8 @@
294
  },
295
  {
296
  "path": "README.md",
297
- "bytes": 17198,
298
- "sha256": "006177225cb135e5f376f7ac7383a93569d30639d3e555c2dfb500d84a04f7d5"
299
  },
300
  {
301
  "path": "requirements-tested.txt",
 
89
  },
90
  {
91
  "path": "docs/EVALUATION.md",
92
+ "bytes": 12599,
93
+ "sha256": "b350f2990dc6a192211bd065f5f28920cb513507981f42d76e0ad0bc1e6e724b"
94
  },
95
  {
96
  "path": "docs/EXPORTS.md",
 
294
  },
295
  {
296
  "path": "README.md",
297
+ "bytes": 18551,
298
+ "sha256": "a5ed283e363b362cebce9576a2e9d2b29ac2989d34e391d9b29d060b1b5170b6"
299
  },
300
  {
301
  "path": "requirements-tested.txt",