Add offline desktop speech with Silero Xenia for Russian
This commit is contained in:
sergeych committed
1 parent
9cd8ebc422
commit
9dbe30b20c
28 files changed
+1865
-16
No files matched your search
@@ -0,0 +1,176 @@
|
||||
# Desktop CPU speech trial
|
||||
|
||||
Desktop Read aloud defaults to **Silero v5_5_ru / Xenia** for Russian when its
|
||||
local runtime is installed. Silero runs in a persistent CPU-only Python process
|
||||
and handles its own stress and ё restoration. English and mixed-script chunks
|
||||
use Supertonic 3 / F1 through ONNX Runtime's CPU provider. Mixed passages retain
|
||||
Supertonic's Russian pronunciation limitations. Plain Supertonic is retained as an audition baseline. Initial listening feedback
|
||||
found its Russian word stress unacceptable. An optional RUAccent CPU stage now
|
||||
adds stress and restores ё before synthesis; its +vowel notation is converted to
|
||||
a Cyrillic vowel followed by U+0301. Manual minimal-pair listening showed stress
|
||||
changes, but this is not a comprehensive pronunciation-quality evaluation.
|
||||
Android retains its existing system TTS implementation.
|
||||
|
||||
## Run
|
||||
|
||||
From the repository root:
|
||||
|
||||
```sh
|
||||
python3 scripts/download-tts-model.py
|
||||
./gradlew :composeApp:run
|
||||
```
|
||||
|
||||
The script downloads about 401 MB into the ignored `.local/tts/supertonic-3`
|
||||
directory. Downloads are revision-pinned and verified against the committed file
|
||||
manifest. It reuses valid files and replaces incomplete downloads only after hash
|
||||
verification. No Python packages are needed. The model license is saved alongside
|
||||
its weights.
|
||||
|
||||
Open a book and choose **Read aloud** from the reader menu. Playback highlights the
|
||||
current sentence, follows the reading position, and exposes Stop, previous/next
|
||||
sentence, and voice settings. Silero offers Xenia (default), Baya, Aidar, Kseniya
|
||||
and Eugene. The Supertonic engines offer F1–F5 and M1–M5. Engine and voice choices
|
||||
last for this application session; each launch starts with Xenia if installed.
|
||||
|
||||
The model loads lazily on first playback. One chunk is prepared ahead while audio
|
||||
plays. Stop, skipping, and voice changes cancel the old playback and lookahead;
|
||||
Supertonic inference checks cancellation between model stages/denoising steps;
|
||||
cancelling Silero inference terminates its worker, which restarts on demand. Long
|
||||
sentences are split at whitespace into at most 300-character chunks. English and
|
||||
Russian are selected from their scripts; mixed-script chunks use the model's
|
||||
language-agnostic `na` mode. This heuristic is not general language detection.
|
||||
|
||||
For a model installed elsewhere, set `YEREED_TTS_MODEL_DIR` (or JVM property
|
||||
`yereed.tts.modelDir`) to the directory containing `onnx/` and `voice_styles/`.
|
||||
Without an override the app searches parent directories for the workspace model,
|
||||
then falls back to `~/.toread/tts/supertonic-3`. Packaged applications can use that
|
||||
home-directory location. The application itself never downloads models or sends
|
||||
text over the network. The first Gradle build and model setup require downloads.
|
||||
|
||||
## Optional RUAccent stage (desktop trial)
|
||||
|
||||
```sh
|
||||
python3 scripts/setup-ruaccent.py
|
||||
./gradlew :composeApp:run
|
||||
```
|
||||
|
||||
This creates a private Python 3.12 runtime under `.local/ruaccent-runtime` and
|
||||
fetches about 726 MB of revision-pinned RUAccent assets into `.local/tts/ruaccent`.
|
||||
It does not change system Python. The bootstrap supports Linux x86_64 directly;
|
||||
on other systems install `uv` first. The dependency versions and model hashes
|
||||
are recorded under `scripts/`. This desktop trial uses RUAccent 1.5.8.3,
|
||||
`turbo3.1`, the full dictionary/rule pipeline, and CPU execution with four threads.
|
||||
This is a desktop quality experiment, not an Android footprint recommendation.
|
||||
|
||||
After setup, restart the reader. **Supertonic 3 + RUAccent** becomes the default
|
||||
engine; select **Supertonic 3** in read-aloud settings for a direct comparison.
|
||||
English bypasses the accentor. Already marked Russian chunks are preserved.
|
||||
The accentor is a persistent local child process communicating over stdin/stdout;
|
||||
Hugging Face and Transformers run in offline mode. Book contents are never sent
|
||||
to a server. Set `YEREED_RUACCENT_HOME` to override its installation directory.
|
||||
Stop/skip kills a pending accentor request when necessary; the next request
|
||||
restarts the worker. Displayed book text and stored files are unchanged.
|
||||
|
||||
The model recognizes U+0301, and preprocessing preserves it. For example,
|
||||
`з+амок` becomes `за́мок`, not a Latin accented-vowel substitution. Predicted
|
||||
stress can still be wrong; model-token support does not guarantee compliance.
|
||||
|
||||
## Audition / benchmark
|
||||
|
||||
```sh
|
||||
./gradlew :composeApp:ttsSample
|
||||
# Also play the four samples through the desktop audio device:
|
||||
./gradlew :composeApp:ttsSample -PttsPlay=true
|
||||
# Controlled manual stress pairs, same words/noise seed:
|
||||
./gradlew :composeApp:ttsSample -PttsStress=true -PttsPlay=true
|
||||
# Original plain Russian passage, then RUAccent + Supertonic:
|
||||
./gradlew :composeApp:ttsSample -PttsRuAccent=true -PttsPlay=true
|
||||
```
|
||||
|
||||
WAV files and their exact unaccented texts go to `composeApp/build/tts-samples/`.
|
||||
The sample task uses the same inference and audio code as the reader without
|
||||
opening or changing the library. It prints model-load time, synthesis time,
|
||||
audio duration, and real-time factor (generation seconds / audio seconds).
|
||||
|
||||
Initial measurements on an Intel Core Ultra 9 275HX, four CPU threads, eight
|
||||
steps, speed 1.05: 1.075 seconds model load; English/Russian F1/M1 samples generated
|
||||
11.6–15.3 seconds of audio in 2.4–3.3 seconds (RTF 0.20–0.24). These are short
|
||||
local samples, not phone/battery or long-book benchmarks. Synthesis completion
|
||||
and non-silent waveforms do not establish pronunciation quality.
|
||||
|
||||
## Dependencies and provenance
|
||||
|
||||
- [Supertonic source](https://github.com/supertone-oss-archive/supertonic): the
|
||||
Kotlin inference code adapts the MIT Java example; attribution is in
|
||||
[licenses/supertonic-code-MIT.txt](licenses/supertonic-code-MIT.txt).
|
||||
- [Supertonic 3 model](https://huggingface.co/supertone-oss-archive/supertonic-3):
|
||||
revision `aafc6e32416a594460b32413efc49d7fe4ce6d46`, OpenRAIL-M weights.
|
||||
Upstream development/support ended in September 2026.
|
||||
- ONNX Runtime 1.23.1, standard CPU artifact; no CUDA/GPU provider.
|
||||
- JSON loading uses Jackson; PCM playback uses Java Sound.
|
||||
|
||||
Run `./gradlew :composeApp:jvmTest` for text-boundary, PCM conversion, queue,
|
||||
lookahead, cancellation, voice-change, and missing-model/error tests, together
|
||||
with existing desktop/common tests. Model inference is exercised separately by
|
||||
`ttsSample`, so ordinary tests do not download weights or use speakers.
|
||||
|
||||
The isolated virtual-display capture did not provide a usable UI image; rendered
|
||||
GUI verification remains incomplete. No further screen access is used.
|
||||
|
||||
RUAccent measurements on the same host: about 5.9 seconds cold load, 30–43 ms
|
||||
for the two warm short passages, and about 2.1 GiB resident memory for the full
|
||||
accentor worker alone. The combined passage took 2.98 seconds of TTS synthesis
|
||||
for 16.42 seconds of audio, excluding accentor startup. In that passage RUAccent
|
||||
incorrectly chose `за́мок` for the door lock; on a shorter explicit door-lock
|
||||
sentence it chose `замо́к`. These are limited observations, not an accuracy score.
|
||||
|
||||
## Silero Russian audition
|
||||
|
||||
User listening feedback rejected Supertonic's Russian quality. A separate CPU
|
||||
audition uses official Silero `v5_5_ru`, with its own stress, ё, stress-homograph
|
||||
and ё-homograph processing enabled. It does not use RUAccent. The same model is
|
||||
now the preferred desktop Russian engine. The Russian model is CC BY-NC-SA 4.0, unlike Silero's MIT CIS
|
||||
base models; see [upstream](https://github.com/snakers4/silero-models).
|
||||
|
||||
Setup from the repository root (requires `uv`; our existing local copy is
|
||||
`.local/tools/uv`):
|
||||
|
||||
```sh
|
||||
export UV_CACHE_DIR="$PWD/.local/uv-cache"
|
||||
export UV_PYTHON_INSTALL_DIR="$PWD/.local/python"
|
||||
export UV_SYSTEM_CERTS=true
|
||||
.local/tools/uv venv --python 3.12.14 .local/silero-runtime
|
||||
.local/tools/uv pip sync --python .local/silero-runtime/bin/python scripts/silero-requirements.txt --index-url https://download.pytorch.org/whl/cpu
|
||||
mkdir -p .local/tts/silero-v5_5-ru
|
||||
curl -fL https://models.silero.ai/models/tts/ru/v5_5_ru.pt -o .local/tts/silero-v5_5-ru/model.pt.part
|
||||
mv .local/tts/silero-v5_5-ru/model.pt.part .local/tts/silero-v5_5-ru/model.pt
|
||||
.local/silero-runtime/bin/python scripts/silero-sample.py
|
||||
python3 - <<'PY'
|
||||
from pathlib import Path
|
||||
import json
|
||||
root = Path.cwd()
|
||||
(root / '.local/tts/silero-v5_5-ru/runtime.json').write_text(json.dumps({
|
||||
'python': str(root / '.local/silero-runtime/bin/python')
|
||||
}) + '\n')
|
||||
PY
|
||||
# Restart the reader after setup, or check its actual Kotlin/Python bridge:
|
||||
./gradlew :composeApp:ttsSample -PttsSilero=true
|
||||
```
|
||||
|
||||
Outputs: `composeApp/build/tts-samples/silero-*.wav`, matching input `.txt`
|
||||
files, and `silero-benchmark.json` with timings, settings and the downloaded
|
||||
model's SHA-256. The script uses CPU-only PyTorch, four threads and 48 kHz PCM.
|
||||
Default voices are Xenia, Baya and Aidar; `--voices xenia` limits the audition.
|
||||
Each reads the earlier library passage and a separate unaccented ё/context
|
||||
challenge. Pronunciation quality must be judged by listening.
|
||||
|
||||
The desktop integration discovers `silero-v5_5-ru` beside the Supertonic model
|
||||
directory; `YEREED_SILERO_HOME` can override it. `runtime.json` points to the
|
||||
isolated Python executable. The worker script is bundled in the application;
|
||||
model loading and inference are local. Both models are needed for bilingual use.
|
||||
|
||||
Initial audition on the same desktop: model load 0.38 seconds; six samples
|
||||
generated 13.2–15.5 seconds of audio in 0.36–0.82 seconds (RTF 0.027–0.059,
|
||||
including the first synthesis). All outputs were finite and non-silent.
|
||||
Downloaded model SHA-256:
|
||||
`50081637b602126ee06cb3bc8a744d25651d2da149ee8864b9a379bfdd934437`.
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2025 Supertone Inc.
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
Reference in new issue
Block a user