toread/docs/desktop-tts.md

# Desktop CPU speech trial

Desktop Read aloud defaults to **Silero v5_5_ru / Xenia** for Russian when its
local runtime is installed. Silero runs in a persistent CPU-only Python process
and handles its own stress and ё restoration. English and mixed-script chunks
use Supertonic 3 / F1 through ONNX Runtime's CPU provider. Mixed passages retain
Supertonic's Russian pronunciation limitations. Plain Supertonic is retained as an audition baseline. Initial listening feedback
found its Russian word stress unacceptable. An optional RUAccent CPU stage now
adds stress and restores ё before synthesis; its +vowel notation is converted to
a Cyrillic vowel followed by U+0301. Manual minimal-pair listening showed stress
changes, but this is not a comprehensive pronunciation-quality evaluation.
Android retains its existing system TTS implementation.

## Run

From the repository root:

```sh
python3 scripts/download-tts-model.py
./gradlew :composeApp:run
```

The script downloads about 401 MB into the ignored `.local/tts/supertonic-3`
directory. Downloads are revision-pinned and verified against the committed file
manifest. It reuses valid files and replaces incomplete downloads only after hash
verification. No Python packages are needed. The model license is saved alongside
its weights.

Open a book and choose **Read aloud** from the reader menu. Playback highlights the
current sentence, follows the reading position, and exposes Stop, previous/next
sentence, and voice settings. Silero offers Xenia (default), Baya, Aidar, Kseniya
and Eugene. The Supertonic engines offer F1–F5 and M1–M5. Engine and voice choices
last for this application session; each launch starts with Xenia if installed.

The model loads lazily on first playback. One chunk is prepared ahead while audio
plays. Stop, skipping, and voice changes cancel the old playback and lookahead;
Supertonic inference checks cancellation between model stages/denoising steps;
cancelling Silero inference terminates its worker, which restarts on demand. Long
sentences are split at whitespace into at most 300-character chunks. English and
Russian are selected from their scripts; mixed-script chunks use the model's
language-agnostic `na` mode. This heuristic is not general language detection.

For a model installed elsewhere, set `YEREED_TTS_MODEL_DIR` (or JVM property
`yereed.tts.modelDir`) to the directory containing `onnx/` and `voice_styles/`.
Without an override the app searches parent directories for the workspace model,
then falls back to `~/.toread/tts/supertonic-3`. Packaged applications can use that
home-directory location. The application itself never downloads models or sends
text over the network. The first Gradle build and model setup require downloads.

## Optional RUAccent stage (desktop trial)

```sh
python3 scripts/setup-ruaccent.py
./gradlew :composeApp:run
```

This creates a private Python 3.12 runtime under `.local/ruaccent-runtime` and
fetches about 726 MB of revision-pinned RUAccent assets into `.local/tts/ruaccent`.
It does not change system Python. The bootstrap supports Linux x86_64 directly;
on other systems install `uv` first. The dependency versions and model hashes
are recorded under `scripts/`. This desktop trial uses RUAccent 1.5.8.3,
`turbo3.1`, the full dictionary/rule pipeline, and CPU execution with four threads.
This is a desktop quality experiment, not an Android footprint recommendation.

After setup, restart the reader. **Supertonic 3 + RUAccent** becomes the default
engine; select **Supertonic 3** in read-aloud settings for a direct comparison.
English bypasses the accentor. Already marked Russian chunks are preserved.
The accentor is a persistent local child process communicating over stdin/stdout;
Hugging Face and Transformers run in offline mode. Book contents are never sent
to a server. Set `YEREED_RUACCENT_HOME` to override its installation directory.
Stop/skip kills a pending accentor request when necessary; the next request
restarts the worker. Displayed book text and stored files are unchanged.

The model recognizes U+0301, and preprocessing preserves it. For example,
`з+амок` becomes `за́мок`, not a Latin accented-vowel substitution. Predicted
stress can still be wrong; model-token support does not guarantee compliance.

## Audition / benchmark

```sh
./gradlew :composeApp:ttsSample
# Also play the four samples through the desktop audio device:
./gradlew :composeApp:ttsSample -PttsPlay=true
# Controlled manual stress pairs, same words/noise seed:
./gradlew :composeApp:ttsSample -PttsStress=true -PttsPlay=true
# Original plain Russian passage, then RUAccent + Supertonic:
./gradlew :composeApp:ttsSample -PttsRuAccent=true -PttsPlay=true
```

WAV files and their exact unaccented texts go to `composeApp/build/tts-samples/`.
The sample task uses the same inference and audio code as the reader without
opening or changing the library. It prints model-load time, synthesis time,
audio duration, and real-time factor (generation seconds / audio seconds).

Initial measurements on an Intel Core Ultra 9 275HX, four CPU threads, eight
steps, speed 1.05: 1.075 seconds model load; English/Russian F1/M1 samples generated
11.6–15.3 seconds of audio in 2.4–3.3 seconds (RTF 0.20–0.24). These are short
local samples, not phone/battery or long-book benchmarks. Synthesis completion
and non-silent waveforms do not establish pronunciation quality.

## Dependencies and provenance

- [Supertonic source](https://github.com/supertone-oss-archive/supertonic): the
  Kotlin inference code adapts the MIT Java example; attribution is in
  [licenses/supertonic-code-MIT.txt](licenses/supertonic-code-MIT.txt).
- [Supertonic 3 model](https://huggingface.co/supertone-oss-archive/supertonic-3):
  revision `aafc6e32416a594460b32413efc49d7fe4ce6d46`, OpenRAIL-M weights.
  Upstream development/support ended in September 2026.
- ONNX Runtime 1.23.1, standard CPU artifact; no CUDA/GPU provider.
- JSON loading uses Jackson; PCM playback uses Java Sound.

Run `./gradlew :composeApp:jvmTest` for text-boundary, PCM conversion, queue,
lookahead, cancellation, voice-change, and missing-model/error tests, together
with existing desktop/common tests. Model inference is exercised separately by
`ttsSample`, so ordinary tests do not download weights or use speakers.

The isolated virtual-display capture did not provide a usable UI image; rendered
GUI verification remains incomplete. No further screen access is used.

RUAccent measurements on the same host: about 5.9 seconds cold load, 30–43 ms
for the two warm short passages, and about 2.1 GiB resident memory for the full
accentor worker alone. The combined passage took 2.98 seconds of TTS synthesis
for 16.42 seconds of audio, excluding accentor startup. In that passage RUAccent
incorrectly chose `за́мок` for the door lock; on a shorter explicit door-lock
sentence it chose `замо́к`. These are limited observations, not an accuracy score.

## Silero Russian audition

User listening feedback rejected Supertonic's Russian quality. A separate CPU
audition uses official Silero `v5_5_ru`, with its own stress, ё, stress-homograph
and ё-homograph processing enabled. It does not use RUAccent. The same model is
now the preferred desktop Russian engine. The Russian model is CC BY-NC-SA 4.0, unlike Silero's MIT CIS
base models; see [upstream](https://github.com/snakers4/silero-models).

Setup from the repository root (requires `uv`; our existing local copy is
`.local/tools/uv`):

```sh
export UV_CACHE_DIR="$PWD/.local/uv-cache"
export UV_PYTHON_INSTALL_DIR="$PWD/.local/python"
export UV_SYSTEM_CERTS=true
.local/tools/uv venv --python 3.12.14 .local/silero-runtime
.local/tools/uv pip sync --python .local/silero-runtime/bin/python scripts/silero-requirements.txt --index-url https://download.pytorch.org/whl/cpu
mkdir -p .local/tts/silero-v5_5-ru
curl -fL https://models.silero.ai/models/tts/ru/v5_5_ru.pt -o .local/tts/silero-v5_5-ru/model.pt.part
mv .local/tts/silero-v5_5-ru/model.pt.part .local/tts/silero-v5_5-ru/model.pt
.local/silero-runtime/bin/python scripts/silero-sample.py
python3 - <<'PY'
from pathlib import Path
import json
root = Path.cwd()
(root / '.local/tts/silero-v5_5-ru/runtime.json').write_text(json.dumps({
    'python': str(root / '.local/silero-runtime/bin/python')
}) + '\n')
PY
# Restart the reader after setup, or check its actual Kotlin/Python bridge:
./gradlew :composeApp:ttsSample -PttsSilero=true
```

Outputs: `composeApp/build/tts-samples/silero-*.wav`, matching input `.txt`
files, and `silero-benchmark.json` with timings, settings and the downloaded
model's SHA-256. The script uses CPU-only PyTorch, four threads and 48 kHz PCM.
Default voices are Xenia, Baya and Aidar; `--voices xenia` limits the audition.
Each reads the earlier library passage and a separate unaccented ё/context
challenge. Pronunciation quality must be judged by listening.

The desktop integration discovers `silero-v5_5-ru` beside the Supertonic model
directory; `YEREED_SILERO_HOME` can override it. `runtime.json` points to the
isolated Python executable. The worker script is bundled in the application;
model loading and inference are local. Both models are needed for bilingual use.

Initial audition on the same desktop: model load 0.38 seconds; six samples
generated 13.2–15.5 seconds of audio in 0.36–0.82 seconds (RTF 0.027–0.059,
including the first synthesis). All outputs were finite and non-silent.
Downloaded model SHA-256:
`50081637b602126ee06cb3bc8a744d25651d2da149ee8864b9a379bfdd934437`.