fix(engines): Confucius4-TTS validated E2E — clone sys.path import, 22.05 kHz, real install docs (#590) (#872)

Co-authored-by: mergetest <test@local>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Palash Debnath
2026-07-02 04:49:50 +05:30
committed by GitHub
co-authored by mergetest Claude Fable 5
parent ae516cae63
commit 4eed552153
9 changed files with 175 additions and 91 deletions
+11
View File
@@ -10,6 +10,17 @@ The bundled TTS model package (`pyproject.toml`) is versioned independently.
### Fixed
- **Confucius4-TTS is now validated end-to-end — and actually loads.** The
opt-in engine's first live run (Apple Silicon, CPU) caught three
scaffold-era faults: the sidecar could never import `confuciustts` (upstream
ships no packaging, so the documented `pip install -e` fails — the sidecar
and bootstrap probe now put the clone on `sys.path`, like upstream's own
example), the assumed 24 kHz sample rate was wrong (confirmed **22 050 Hz**,
now regression-tested), and the docs demanded an Amphion/MaskGCT install
that doesn't exist (all weights auto-download from HuggingFace). CPU is
~17× realtime, so CUDA stays the recommended path; `gpu_compat` now
advertises `("cuda", "cpu")`. (#590)
- **Parakeet TDT transcription now works without an NVIDIA GPU.** The
`nemo-parakeet` ASR engine (parakeet-tdt-0.6b-v3, 25 languages, word
timestamps) was hard-gated behind CUDA — but a live measurement on an Apple
+2 -2
View File
@@ -292,11 +292,11 @@ OmniVoice ships a multi-engine TTS backend. The default engine (OmniVoice) is al
| **Supertonic 3** ⚡ | 31 | — | — | ✅ CPU | ✅ CPU | ✅ CPU | OpenRAIL-M |
| **MOSS-TTS-v1.5** ⚡ (8B) | 31 | ✅ | — | ✅ CUDA/CPU | ✅ CPU | ✅ CUDA/CPU | Apache-2.0 |
| **dots.tts** ⚡ (2B) | 24 | ✅ | — | ✅ CUDA/CPU | ✅ CPU | ❌ | Apache-2.0 |
| **Confucius4-TTS** ⚡ | 14 | ✅ | — | ✅ CUDA | — | ✅ CUDA | Apache-2.0 |
| **Confucius4-TTS** ⚡ | 14 | ✅ | — | ✅ CUDA/CPU | ✅ CPU | ✅ CUDA/CPU | Apache-2.0 |
> **CUDA** = GPU-accelerated · **MPS** = Apple Silicon Metal · **CPU** = runs everywhere, slower for large models · KittenTTS and MOSS-TTS-Nano run realtime on CPU · MLX-Audio is Apple Silicon only · ⚡ = lazy-registered (installed on first use)
>
> **MOSS-TTS-v1.5** (8B, ~16 GB weights) and **dots.tts** (2B, ~9 GB weights) are heavyweight opt-in engines that run in their own isolated venv from a local clone — see [MOSS-TTS-v1.5](docs/engines/moss-tts-v15.md) and [dots.tts](docs/engines/dots-tts.md). Neither claims Apple-Silicon **MPS** (upstream is CUDA/CPU only; on a Mac they run on CPU). dots.tts upstream is Linux/macOS only — no Windows path. **Confucius4-TTS** (14-language cross-lingual zero-shot cloning) is similar — its own Python 3.10/CUDA venv from a clone, NVIDIA-only; see [Confucius4-TTS](docs/engines/confucius4-tts.md). ⚠️ scaffold — needs on-hardware validation before production use.
> **MOSS-TTS-v1.5** (8B, ~16 GB weights) and **dots.tts** (2B, ~9 GB weights) are heavyweight opt-in engines that run in their own isolated venv from a local clone — see [MOSS-TTS-v1.5](docs/engines/moss-tts-v15.md) and [dots.tts](docs/engines/dots-tts.md). Neither claims Apple-Silicon **MPS** (upstream is CUDA/CPU only; on a Mac they run on CPU). dots.tts upstream is Linux/macOS only — no Windows path. **Confucius4-TTS** (14-language cross-lingual zero-shot cloning) is similar — its own Python 3.10 venv from a clone; CUDA recommended, CPU validated end-to-end (slow, ~17× realtime; no MPS — tested slower than CPU); see [Confucius4-TTS](docs/engines/confucius4-tts.md).
### ASR Engines
+22 -18
View File
@@ -10,12 +10,13 @@ parent. It is **opt-in** — selected in the engine picker and enabled only when
the user points ``OMNIVOICE_CONFUCIUS4_TTS_DIR`` at a clone so it can never
become a broken default on any platform (the strict default-parity rule).
Status (#590): the synthesis API (``confuciustts.cli.inference.ConfuciusTTS`` →
``.generate(text, lang, prompt_wav)`` tensor, ``model.sample_rate``) is
validated against the upstream repo, and the sidecar's pure logic is unit-tested
(``tests/test_confucius4_sidecar.py``). A one-time end-to-end run on a CUDA 12.6
GPU is still needed to confirm the live model call and the true output sample
rate. Gated off by default, so this affects no one until they opt in.
Status (#590): **validated end-to-end** (2026-07-02, Apple Silicon, CPU) — the
synthesis API (``confuciustts.cli.inference.ConfuciusTTS``
``.generate(text, lang, prompt_wav)`` tensor, ``model.sample_rate``) produced
audible speech at 22 050 Hz; the sidecar's pure logic is unit-tested
(``tests/test_confucius4_sidecar.py``). CPU inference is slow (~17× realtime),
so CUDA is the recommended path. Gated off by default, so this affects no one
until they opt in.
Three entry points: ``Confucius4Backend`` (this module), ``main.py`` (the sidecar,
runs under the Confucius4 venv never imported by the parent), and
@@ -35,7 +36,7 @@ logger = logging.getLogger("omnivoice.confucius4")
class Confucius4Backend(SubprocessBackend):
"""Confucius4-TTS (netease-youdao) — LLM-based, 14 langs, zero-shot clone, CUDA.
"""Confucius4-TTS (netease-youdao) — LLM-based, 14 langs, zero-shot clone.
Runs in a long-lived sidecar over length-prefixed JSON-over-stdio in a
dedicated venv. First synthesize cold-loads the checkpoint; subsequent calls
@@ -45,22 +46,25 @@ class Confucius4Backend(SubprocessBackend):
git clone https://github.com/netease-youdao/Confucius4-TTS.git
cd Confucius4-TTS
uv venv --python 3.10 && uv pip install -r requirements.txt && uv pip install -e .
uv venv --python 3.10 && uv pip install -r requirements.txt
(Upstream ships no pyproject.toml/setup.py, so there is nothing to
``pip install -e`` the sidecar sys.path-inserts the clone instead.)
Then set ``OMNIVOICE_CONFUCIUS4_TTS_DIR`` to the clone root and restart.
License: Apache-2.0. Requires an NVIDIA GPU (CUDA 12.6); no MPS / CPU path.
License: Apache-2.0. CUDA recommended; CPU validated but ~17× realtime.
"""
id = "confucius4-tts"
display_name = (
"Confucius4-TTS (LLM, 14 langs, cross-lingual zero-shot clone, CUDA, Apache-2.0)"
"Confucius4-TTS (LLM, 14 langs, cross-lingual zero-shot clone, CUDA/CPU, Apache-2.0)"
)
supports_voice_design = False # timbre comes from a reference clip
# Upstream vocoder sample rate is not documented; re-read from the sidecar's
# ready/audio frames. 24 kHz is the conservative default until confirmed.
_DEFAULT_SAMPLE_RATE = 24000
# NVIDIA/CUDA only — no MPS or CPU branch documented upstream.
gpu_compat = ("cuda",)
# Upstream vocoder rate (config target_sample_rate) — confirmed 22 050 Hz by
# a live run (2026-07-02); still re-read from the sidecar's ready/audio frames.
_DEFAULT_SAMPLE_RATE = 22050
# CUDA fast path + CPU fallback, both exercised (CPU end-to-end validated).
# No MPS claim — upstream has no Metal path.
gpu_compat = ("cuda", "cpu")
@classmethod
def is_available(cls) -> tuple[bool, str]:
@@ -75,15 +79,15 @@ class Confucius4Backend(SubprocessBackend):
return False, (
"Confucius4-TTS venv not found. Set OMNIVOICE_CONFUCIUS4_TTS_DIR "
"to your Confucius4-TTS clone (the directory containing "
"requirements.txt) and restart OmniVoice. NVIDIA GPU (CUDA) "
"required. See docs/engines/confucius4-tts.md."
"requirements.txt) and restart OmniVoice. CUDA GPU recommended "
"(CPU works but is slow). See docs/engines/confucius4-tts.md."
)
if not CONFUCIUS4_SIDECAR_SCRIPT.exists():
return False, (
"Confucius4-TTS sidecar script missing at "
f"{CONFUCIUS4_SIDECAR_SCRIPT} — reinstall OmniVoice."
)
return True, "ok (CUDA)"
return True, "ok"
@classmethod
def venv_python(cls):
+24 -11
View File
@@ -11,13 +11,13 @@ Probe order (existing power-user installs win — zero migration):
1. ``${OMNIVOICE_CONFUCIUS4_TTS_DIR}/.venv/`` the user's clone-level venv.
2. ``backend/engines/confucius4/.venv/`` this package's own venv.
3. Bootstrap: ``uv venv`` then ``uv pip install -r <clone>/requirements.txt``
+ ``uv pip install -e <clone>`` (the upstream stack).
(+ ``uv pip install -e <clone>`` only if upstream ever ships packaging).
Scaffold (#590): the install/import details mirror the documented upstream
layout but have NOT been validated on hardware confirm the package name
(``confuciustts``) and requirements path against your clone before relying on
the auto-bootstrap. The engine is opt-in (env-dir gated) and never touched
unless ``OMNIVOICE_CONFUCIUS4_TTS_DIR`` is set, so this can't affect the default
Validated end-to-end 2026-07-02 (Apple Silicon, CPU): upstream ships **no
pyproject.toml/setup.py**, so ``confuciustts`` is importable only with the
clone root on ``sys.path`` the import probe and the sidecar both handle
that. The engine is opt-in (env-dir gated) and never touched unless
``OMNIVOICE_CONFUCIUS4_TTS_DIR`` is set, so this can't affect the default
install on any platform.
"""
from __future__ import annotations
@@ -113,11 +113,20 @@ def _probe_paths() -> list[Path]:
return out
def _import_probe_code() -> str:
"""Probe snippet mirroring the sidecar's import semantics: upstream is not
pip-installable, so ``confuciustts`` resolves via the clone on sys.path."""
clone = os.environ.get(_CLONE_DIR_ENV, "")
if clone:
return f"import sys; sys.path.insert(0, {clone!r}); import {_IMPORT_PROBE}"
return f"import {_IMPORT_PROBE}"
def _venv_can_import(python_path: Path) -> bool:
"""Spawn the candidate python and verify ``import confuciustts`` works."""
try:
proc = subprocess.run(
[str(python_path), "-c", f"import {_IMPORT_PROBE}"],
[str(python_path), "-c", _import_probe_code()],
capture_output=True, timeout=_IMPORT_PROBE_TIMEOUT_S,
)
except (subprocess.TimeoutExpired, OSError) as exc:
@@ -173,10 +182,14 @@ def _bootstrap_engines_venv(clone_dir: Path) -> Path:
"-r", str(requirements)],
check=True, timeout=_UV_PIP_INSTALL_TIMEOUT_S, capture_output=True,
)
subprocess.run(
[uv, "pip", "install", "--python", str(python_path), "-e", str(clone_dir)],
check=True, timeout=_UV_PIP_INSTALL_TIMEOUT_S, capture_output=True,
)
# Editable install only if upstream ever ships packaging metadata —
# as of 2026-07 there is none, and `uv pip install -e` on a bare clone
# fails outright. Import resolution is handled via sys.path instead.
if (clone_dir / "pyproject.toml").is_file() or (clone_dir / "setup.py").is_file():
subprocess.run(
[uv, "pip", "install", "--python", str(python_path), "-e", str(clone_dir)],
check=True, timeout=_UV_PIP_INSTALL_TIMEOUT_S, capture_output=True,
)
except subprocess.CalledProcessError as exc:
raise RuntimeError(
"uv pip install failed during Confucius4 bootstrap "
+25 -8
View File
@@ -18,10 +18,10 @@ Op flow: ready → ping/pong → synthesize (→ progress, → audio) → shutdo
Status (#590): the model API below
(``confuciustts.cli.inference.ConfuciusTTS(config_path=, device=)`` and
``model.generate(text=, lang=, prompt_wav=)`` audio tensor, ``model.sample_rate``)
is validated against the upstream repo; this sidecar's pure logic is unit-tested
in ``tests/test_confucius4_sidecar.py``. Still pending a one-time CUDA-12.6 GPU
run to confirm the live model call + true sample rate. Opt-in, so it affects no
one until enabled.
is **validated end-to-end** (2026-07-02, Apple Silicon, CPU): live generate()
produced audible speech at 22 050 Hz. This sidecar's pure logic is unit-tested
in ``tests/test_confucius4_sidecar.py``. Opt-in, so it affects no one until
enabled.
Restrictions: NO imports from OmniVoice parent code. NO logging of os.environ.
"""
@@ -36,9 +36,11 @@ import traceback
MAX_FRAME_BYTES = 64 * 1024 * 1024
#: Conservative default until the upstream vocoder rate is confirmed; the real
#: value is re-read from each generate() result if the model exposes it.
CONFUCIUS_SAMPLE_RATE = 24000
#: Upstream BigVGAN vocoder rate — ``target_sample_rate: 22050`` in
#: ``config/inference_config.yaml``, confirmed by a live end-to-end run
#: (2026-07-02). The real value is still re-read from ``model.sample_rate``
#: on each generate() so a future upstream change can't corrupt audio.
CONFUCIUS_SAMPLE_RATE = 22050
def _send(stream, obj: dict) -> None:
@@ -87,14 +89,29 @@ def _config_path() -> str:
return os.path.join(clone, "config", "inference_config.yaml")
def _ensure_clone_on_sys_path() -> None:
"""Make ``import confuciustts`` resolve from the user's clone.
Upstream Confucius4-TTS is **not pip-installable** (no pyproject.toml /
setup.py as of 2026-07); its own ``example.py`` sys.path-inserts the repo
root instead. Mirror that here so the sidecar works from a plain
``uv pip install -r requirements.txt`` venv. Inserted at position 0 so the
clone the user pointed at always wins over any stale installed copy.
"""
clone = os.environ.get("OMNIVOICE_CONFUCIUS4_TTS_DIR", "")
if clone and clone not in sys.path:
sys.path.insert(0, clone)
def _load_model(stdout):
"""Cold-construct the Confucius4 model (CUDA, else CPU as a last resort)."""
"""Cold-construct the Confucius4 model (CUDA, else CPU — both validated)."""
global _model
if _model is not None:
return _model
_send(stdout, {"op": "progress", "stage": "loading_model", "percent": 0})
_ensure_clone_on_sys_path()
import torch
from confuciustts.cli.inference import ConfuciusTTS # type: ignore[import-not-found]
+4 -4
View File
@@ -1160,9 +1160,9 @@ _LAZY_REGISTRY: dict[str, tuple[str, str]] = {
"dots-tts": ("engines.dots_tts", "DotsTTSBackend"),
# Issue #590: Confucius4-TTS (netease-youdao) — LLM-based, 14-language
# cross-lingual zero-shot cloning, Apache-2.0. Opt-in + subprocess-isolated
# (own Python 3.10 + CUDA venv) like the entries above. ⚠️ scaffold: the
# sidecar's synthesis API is README-derived and needs hardware validation;
# it's gated behind OMNIVOICE_CONFUCIUS4_TTS_DIR so it's inert until enabled.
# (own Python 3.10 venv) like the entries above. Validated end-to-end
# 2026-07-02 (CPU, Apple Silicon; 22.05 kHz output). Gated behind
# OMNIVOICE_CONFUCIUS4_TTS_DIR so it's inert until enabled.
"confucius4-tts": ("engines.confucius4", "Confucius4Backend"),
}
@@ -1259,7 +1259,7 @@ _INSTALL_HINTS: dict[str, str] = {
"supertonic3": "uv sync --extra supertonic (CPU-only ONNX, 31 langs, ~400 MB model on first use; OpenRAIL-M model license)",
"moss-tts-v15": "git clone OpenMOSS/MOSS-TTS + set OMNIVOICE_MOSS_TTS_V15_DIR (own venv, transformers==5.0; 8B, ~16 GB weights; CUDA/CPU, no MPS; Apache-2.0)",
"dots-tts": "git clone rednote-hilab/dots.tts + set OMNIVOICE_DOTS_TTS_DIR (own venv, transformers==4.57; 2B, ~9 GB weights; CUDA/CPU, Linux/macOS only — no Windows; Apache-2.0)",
"confucius4-tts":"git clone netease-youdao/Confucius4-TTS + set OMNIVOICE_CONFUCIUS4_TTS_DIR (own Python 3.10/CUDA venv; 14-lang cross-lingual zero-shot clone; NVIDIA GPU required; Apache-2.0)",
"confucius4-tts":"git clone netease-youdao/Confucius4-TTS + set OMNIVOICE_CONFUCIUS4_TTS_DIR (own Python 3.10 venv; 14-lang cross-lingual zero-shot clone; ~5 GB weights auto-download; CUDA/CPU, no MPS; Apache-2.0)",
}
+45 -41
View File
@@ -1,18 +1,16 @@
# Confucius4-TTS (opt-in engine)
> **Status: implementation complete — GPU inference run pending.** The
> integration (engine registration, dedicated-venv bootstrap, sidecar wire
> protocol, opt-in gating) is done, and the sidecar's synthesis API
> (`confuciustts.cli.inference.ConfuciusTTS(config_path, device)`
> `generate(text, lang, prompt_wav)` → tensor, `model.sample_rate`) is
> **validated against the upstream repo** and the sidecar's pure logic
> (language normalization, tensor→PCM, config resolution, wire framing,
> synthesize dispatch with the model mocked) is **unit-tested**
> (`tests/test_confucius4_sidecar.py`). What remains is a one-time **end-to-end
> run on a CUDA 12.6 GPU** to confirm the live model call and the true output
> sample rate — no maintainer GPU box is available yet. The engine is gated
> behind `OMNIVOICE_CONFUCIUS4_TTS_DIR`, so it's completely inert until you opt
> in — it can't affect the default install on any platform.
> **Status: validated end-to-end (2026-07-02).** The integration (engine
> registration, dedicated-venv bootstrap, sidecar wire protocol, opt-in gating)
> is done, the sidecar's pure logic is unit-tested
> (`tests/test_confucius4_sidecar.py`), and a live synthesis run on Apple
> Silicon (CPU) produced audible cloned speech — confirming the model API and
> the true output sample rate of **22 050 Hz**. CUDA is the recommended
> hardware; CPU works but is slow (~17× realtime — roughly 100 s for 6 s of
> audio). MPS also runs but is *slower* than CPU (~64× realtime), so the
> sidecar deliberately never selects it. The engine is gated behind
> `OMNIVOICE_CONFUCIUS4_TTS_DIR`, so it's completely inert until you opt in —
> it can't affect the default install on any platform.
[Confucius4-TTS](https://github.com/netease-youdao/Confucius4-TTS) (netease-youdao)
is an LLM-based multilingual / cross-lingual zero-shot voice-cloning TTS.
@@ -21,8 +19,8 @@ is an LLM-based multilingual / cross-lingual zero-shot voice-cloning TTS.
Indonesian, Italian, Thai, Portuguese, Russian, Malay, Vietnamese.
- **Unconstrained cloning** — no reference transcript required.
- **Cross-lingual voice transfer** — keep one voice across languages.
- **License:** Apache-2.0. **Hardware:** NVIDIA GPU, CUDA 12.6, Python 3.10.
No CPU/MPS path documented; not advertised on Apple Silicon.
- **License:** Apache-2.0. **Hardware:** NVIDIA GPU (CUDA 12.6) recommended;
CPU validated on Apple Silicon but ~17× realtime. Output: 22 050 Hz mono.
Like IndexTTS-2 / MOSS-TTS-v1.5 / dots.tts, it runs in its **own subprocess venv**
so its dependency stack never touches the default OmniVoice interpreter.
@@ -34,53 +32,59 @@ git clone https://github.com/netease-youdao/Confucius4-TTS.git
cd Confucius4-TTS
uv venv --python 3.10
uv pip install -r requirements.txt
uv pip install -e .
```
**External dependencies (per upstream) — needed before first synthesis:**
> Upstream ships **no `pyproject.toml`/`setup.py`**, so there is nothing to
> `pip install -e` — don't try; it fails. The OmniVoice sidecar puts the clone
> on `sys.path` itself (the same thing upstream's `example.py` does).
- **MaskGCT codec** from the [Amphion](https://github.com/open-mmlab/Amphion)
repo — Confucius4's semantic-to-acoustic stage uses it. Follow upstream's
README for the exact vendor/install step.
- **`facebook/w2v-bert-2.0`** (Wav2Vec2-BERT) — pulled from HuggingFace on first
use; make sure your `HF_TOKEN` (Settings → Credentials) is set if rate-limited.
- **Checkpoint** `netease-youdao/Confucius4-TTS` (~24 GB: `t2s_model.safetensors`,
`s2a_model.pt`, `wav2vec2bert_stats.pt`, tokenizer files) into `checkpoints/`.
**Model weights — all fetched automatically from HuggingFace on first
synthesis (~5 GB total, cached in `$HF_HUB_CACHE`):**
These are large and CUDA-only; budget disk + a first-run download.
- `netease-youdao/Confucius4-TTS``t2s_model.safetensors` + `s2a_model.pt`
(the tokenizer + `wav2vec2bert_stats.pt` already ship in the clone's
`checkpoints/`).
- `facebook/w2v-bert-2.0` — semantic feature extractor (~2.3 GB).
- `funasr/campplus` — speaker-style encoder (small).
- `nvidia/bigvgan_v2_22khz_80band_256x` — vocoder (BigVGAN and CAMPPlus
*code* is vendored in the clone's `external/`; no Amphion install needed).
Set your `HF_TOKEN` (Settings → Credentials) if you hit rate limits.
Then point OmniVoice at the clone and restart:
- **macOS/Linux:** `export OMNIVOICE_CONFUCIUS4_TTS_DIR=/path/to/Confucius4-TTS`
- **Windows (PowerShell):** `[Environment]::SetEnvironmentVariable("OMNIVOICE_CONFUCIUS4_TTS_DIR","C:\path\to\Confucius4-TTS","User")`
Select **Confucius4-TTS** in Settings → Engines. First synthesize downloads the
checkpoint from `netease-youdao/Confucius4-TTS` (HuggingFace).
Select **Confucius4-TTS** in Settings → Engines. The first synthesize triggers
the weight downloads above, then generates.
### Optional overrides
- `OMNIVOICE_CONFUCIUS4_CONFIG` — path to `inference_config.yaml` if it isn't at
`<clone>/config/inference_config.yaml`.
## Validation status (for the maintainer)
## Validation record (2026-07-02, Apple Silicon M-series, CPU)
The sidecar (`backend/engines/confucius4/main.py`) uses:
```python
from confuciustts.cli.inference import ConfuciusTTS
model = ConfuciusTTS(config_path=..., device="cuda")
model = ConfuciusTTS(config_path=..., device="cuda") # or "cpu"
audio = model.generate(text=..., lang="en", prompt_wav="ref.wav") # → tensor
sr = model.sample_rate
sr = model.sample_rate # 22050
```
- ✅ **Validated against upstream** — import path/package (`confuciustts`), the
`ConfuciusTTS(config_path, device)` constructor, and `generate(text, lang,
prompt_wav)` → tensor all match the repo.
- ✅ **Sidecar logic unit-tested** (`tests/test_confucius4_sidecar.py`, 22 cases):
language normalization, tensor→PCM (mono/stereo/clip), config-path resolution,
wire framing, and synthesize dispatch with the model mocked.
- ⏳ **Pending a CUDA 12.6 GPU run:** confirm the live `generate()` call end-to-end
and the **true output sample rate** — if it isn't 24 kHz, update
`CONFUCIUS_SAMPLE_RATE` in `main.py` / `_DEFAULT_SAMPLE_RATE` in `__init__.py`.
To validate: set `OMNIVOICE_CONFUCIUS4_TTS_DIR`, select the engine, synthesize
one clip on an NVIDIA box, and confirm audible output.
- ✅ **Live end-to-end run**: English zero-shot clone from a 9.5 s reference —
6.06 s of audible speech (peak 0.85) in 102 s on CPU. `model.sample_rate`
returned **22 050**, matching `target_sample_rate` in
`config/inference_config.yaml`; `CONFUCIUS_SAMPLE_RATE` /
`_DEFAULT_SAMPLE_RATE` are pinned to it (regression-tested).
- ✅ **Not pip-installable upstream** — discovered live; the bootstrap now skips
the editable install unless upstream ships packaging, and both the import
probe and the sidecar resolve `confuciustts` via the clone on `sys.path`.
- ✅ **MPS probed and rejected**: runs, but ~4× slower than CPU (Metal op
fallbacks) — the sidecar selects CUDA when available, else CPU, never MPS.
- ✅ **Sidecar logic unit-tested** (`tests/test_confucius4_sidecar.py`):
language normalization, tensor→PCM (mono/stereo/clip), config-path
resolution, clone sys.path injection, wire framing, synthesize dispatch.
+1 -1
View File
@@ -19,7 +19,7 @@ def test_registered_in_lazy_registry():
def test_backend_class_metadata():
from engines.confucius4 import Confucius4Backend
assert Confucius4Backend.id == "confucius4-tts"
assert Confucius4Backend.gpu_compat == ("cuda",) # CUDA-only, no MPS claim
assert Confucius4Backend.gpu_compat == ("cuda", "cpu") # CPU validated E2E; no MPS claim
assert Confucius4Backend.supports_voice_design is False
+41 -6
View File
@@ -2,12 +2,11 @@
The upstream synthesis API (``confuciustts.cli.inference.ConfuciusTTS``
``generate(text, lang, prompt_wav)`` tensor, ``model.sample_rate``) is
validated against the netease-youdao/Confucius4-TTS repo. Actual audio
generation needs a CUDA 12.6 GPU + the ~2-4 GB checkpoint (Amphion/MaskGCT
codec + w2v-bert-2.0), so it can't run in CI — but the sidecar's *pure* logic
(language normalization, tensorPCM, config-path resolution, wire framing) and
the bootstrap probe are fully testable here, with the model mocked. These pin
that logic so a real GPU run only has to confirm the model call itself.
**validated end-to-end** (2026-07-02, Apple Silicon, CPU): audible speech at
22 050 Hz. Full generation needs ~5 GB of weights, so it can't run in CI — but
the sidecar's *pure* logic (language normalization, tensor→PCM, config-path
resolution, sys.path clone injection, wire framing) and the bootstrap probe are
fully testable here, with the model mocked.
The sidecar is stdlib-only at import time (the model/torch imports are lazy),
so we import it directly without spawning the engine venv.
@@ -85,6 +84,42 @@ def test_pcm_accepts_numpy(sc):
assert (sr, n) == (16000, 3)
# ── Sample rate (confirmed 22 050 Hz by the 2026-07-02 live run) ──────────
def test_sample_rate_constant_is_confirmed_upstream_rate(sc):
# Upstream config target_sample_rate — regression-pins the live-run value
# so the pre-validation 24 000 guess can't come back.
assert sc.CONFUCIUS_SAMPLE_RATE == 22050
def test_sample_rate_lockstep_with_backend_default(sc):
import os as _os, sys as _sys
_sys.path.insert(0, _os.path.join(
_os.path.dirname(_os.path.dirname(_os.path.abspath(__file__))), "backend"))
from engines.confucius4 import Confucius4Backend
assert Confucius4Backend._DEFAULT_SAMPLE_RATE == sc.CONFUCIUS_SAMPLE_RATE
# ── Clone sys.path injection (upstream is not pip-installable) ────────────
def test_clone_dir_inserted_at_sys_path_front(sc, monkeypatch):
import sys as _sys
monkeypatch.setenv("OMNIVOICE_CONFUCIUS4_TTS_DIR", "/clone")
monkeypatch.setattr(_sys, "path", ["existing"])
sc._ensure_clone_on_sys_path()
assert _sys.path[0] == "/clone"
sc._ensure_clone_on_sys_path() # idempotent — no dup
assert _sys.path.count("/clone") == 1
def test_no_sys_path_change_without_clone_dir(sc, monkeypatch):
import sys as _sys
monkeypatch.delenv("OMNIVOICE_CONFUCIUS4_TTS_DIR", raising=False)
monkeypatch.setattr(_sys, "path", ["existing"])
sc._ensure_clone_on_sys_path()
assert _sys.path == ["existing"]
# ── Config path resolution ────────────────────────────────────────────────
def test_config_path_explicit_override(sc, monkeypatch):