Model · ASR post-processing
MiniLM-L12 ASR Paragraph Breaks: paragraphs for dictated text, on-device
Decides word by word where a new paragraph starts in flat speech-to-text output. A classifier, so it cannot change the text. 32 MB in int8.
Which model? This replaces the generative Gemma 3 270M model for the same job: higher F1 (0.85 against 0.78), 32 MB instead of 437 MB in int8, and about 0.65 s per memo on a laptop CPU instead of 23–48 s. See the Gemma 3 270M model →
About the model
Speech recognition returns punctuated, cased text as one block. This model scores every word for whether a new paragraph starts there, and the output is the input words joined with a space or a blank line. It cannot change, add or drop a word, so unlike the Gemma model it needs no fallback check.
It is MiniLM-L12-H384 (33.2M parameters) fine-tuned for word-level token classification. Long texts are read in overlapping 510-token windows. The int8 ONNX file an app ships is 32 MB.
The paragraphing style is mine: short paragraphs, with the opening line and each sign-off line on their own.
Which file should I download?
ONNX int8 34 MB
The one to ship on-device. Same F1 as fp32; differs from PyTorch on 33 of 8,824 test words. paragraphs.py in the repo does the windowing.
Run with ONNX Runtime Download ONNX int8 →ONNX fp32 133 MB
Makes the same decision as PyTorch on all 8,824 test words.
Run with ONNX Runtime Download ONNX fp32 →safetensors (Transformers) 133 MB
AutoModelForTokenClassification in Python; use the same windowing for texts over 510 tokens.
Run with Hugging Face Transformers Download safetensors (Transformers) →Results
r5b, run asr-paragraph-r5b-minilm-l12, evaluated 24 October 2026. The same 26 held-out synthetic voice memos (290 hand-reviewed breaks) as the Gemma model, so the figures compare directly. Break-position precision and recall; higher is better. Threshold and epoch were chosen on a separate dev split.
| Test set | Gemma 3 270M r2 | MiniLM-L12 r5b |
|---|---|---|
| Break precision | 0.84 | 0.86 |
| Break recall | 0.72 | 0.84 |
| F1 | 0.78 | 0.85 |
| Fallbacks | 2 of 26 | none possible |
- int8 ONNX: precision 0.86, recall 0.83, F1 0.85. Mean F1 over 3 training seeds: 0.85.
- Segment-any-text (sat-3l-sm, paragraph mode) scored F1 0.65 on the same memos (precision 0.48, recall 1.00).
- On 10 harder hand-written cases (fused greetings, dropped commas, fillers) F1 was 0.83; that set also chose between two tied runs, so it is not a clean held-out figure.
- Latency: 0.65 s mean for a 340-word memo, int8 on a Ryzen 7 5700U, one thread. Phone latency not measured yet.
Training
- Word-level token classification on microsoft/MiniLM-L12-H384-uncased: label 1 where a paragraph starts (first sub-token only), class weight 3 on breaks (about 3% of words), 10 epochs, learning rate 8e-5, batch 16, best dev epoch of 3 seeds. About 140 s per seed on one NVIDIA L4.
- Data: danielrosehill/ASR-Paragraph-Breaks, synthetic spoken-register memos written by Claude Haiku 5.5, 20 hand-reviewed and 784 labelled by rules derived from them. 30% of rows are repeated with ASR noise (lower case after a period, dropped commas, fillers, fused greetings). 936 training rows, 80 dev.
Training data
Checkpoints and roadmap
Where the model is now and the checkpoints still to come. Planned stages are the current plan, not promises.
- r5b Done 24 October 2026
First release: F1 0.85 (int8 0.85), ONNX fp32 and int8.
- Phone latency Planned
Measure int8 inference on Android.
Limitations
- English only, trained and tested on synthetic memos in one person's paragraphing style. The 26-memo test set is small: one break moves F1 by about 0.003.
- It reads the whole text, so it runs once after transcription, not while dictating.
- It still keeps "Hi John," joined to the next sentence in some memos, and can break after a period followed by a lower-case word.