docs+test: finish the RTX 50-series story (#1931)
The code half of #1931 landed already: `torchaudio.set_audio_backend()` is guarded, so the torch 2.9.x upgrade a Blackwell card needs no longer trades one `ml_imports` crash for another. Two things were still missing. The changelog said the upgrade was documented. It was not — nothing in docs/ mentions sm_120, Blackwell, or the 50-series at all, so a user hitting a native access violation inside `import torch` had the issue thread and nothing else. troubleshooting.md now carries it: why the pinned torch 2.8.0 cannot work (no sm_120 kernels in the wheel — not a setting, not a workaround), the trio that has to move together, the verification command that proves the kernels arrived, and the fact that the change is to the repo's own pins so a later pull will undo it. The part most likely to be missed is that there are TWO pin lists. `constraint-dependencies` governs `uv sync`/`uv lock`/`uv run`; deploy/torch-constraints.txt governs the `uv pip install` paths, which ignore project-level uv settings. Editing one leaves the other behind, which is what `RuntimeError: operator torchvision::nms does not exist` looks like from the outside. Both are named. The second gap: nothing protected the guard. CI runs the pinned torch 2.8.0, where `set_audio_backend` still exists, so deleting the `hasattr` as a "simplify this no-op" cleanup would pass every test in the suite and restore a hard startup crash for every RTX 50-series user. tests/ now walks the backend AST and fails on any reach for a torchaudio API that 2.9 removed unless something proves it is there — a `hasattr`/`getattr` check or a `try`. Fails with the guard removed, passes with it. Not addressed here, because it is already fixed: the reporter's third observation, that launching through the desktop shell hung inside `import torch`'s native init, is the OpenBLAS/stdin-pipe deadlock closed under #1952. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017ypcgSsh5j2PEonSJiAU1S
This commit is contained in:
co-authored by
Claude Opus 5
parent
2bf922d9e5
commit
14d6b90836
@@ -85,6 +85,8 @@ the frozen-backend fallback mirror it for their toolchains.
|
||||
|
||||
### Docs
|
||||
|
||||
- The torch upgrade an RTX 50-series card needs is written down, with the second pin file the resolver checks and the command that proves the kernels are there (#1931)
|
||||
|
||||
- Docker quick starts now explain the AMD64-only images and direct Apple Silicon users to the native macOS app (#1921) — thanks @yangfan-yf-yf!
|
||||
- audio.cpp (Breeze-TTS-2) is now a documented opt-in engine: prebuilt binary install, explicit GGUF download, voice modes, and the weights' research/non-commercial terms (#1891)
|
||||
- `docs/STRUCTURE.md` describes the tree as it is today, and a test now keeps its counts honest (#1981) — thanks @Dawcraft!
|
||||
|
||||
@@ -248,6 +248,70 @@ peak memory footprint that exceeds free VRAM. Windows-only quirk.
|
||||
|
||||
**Linked issue:** [#65](https://github.com/debpalash/VoiceStudio/issues/65)
|
||||
|
||||
## 5b. RTX 50-series (Blackwell, sm_120): backend crashes during `ml_imports`
|
||||
|
||||
**Symptom:** on an RTX 5070 / 5070 Ti / 5080 / 5090, the backend never becomes
|
||||
ready. The desktop app sits on "starting backend", `/health` returns 503, and
|
||||
`/startup/progress` shows `ml_imports` active. From source you see `import
|
||||
torch` die with a native access violation rather than a Python traceback.
|
||||
|
||||
**Cause:** VoiceStudio pins `torch 2.8.0`. That build carries no `sm_120`
|
||||
kernels, so on a Blackwell card the CUDA initializer faults inside the native
|
||||
library. This is not a VoiceStudio bug and no setting works around it — the
|
||||
wheel does not contain code for the GPU.
|
||||
|
||||
**Fix:** move the whole torch trio to a build with `sm_120` kernels. They must
|
||||
move together — upgrading one past the ABI the others were built against gives
|
||||
you `RuntimeError: operator torchvision::nms does not exist`, which is the
|
||||
next section's problem instead.
|
||||
|
||||
Edit **both** pin lists, keeping them identical:
|
||||
|
||||
- `[tool.uv] constraint-dependencies` in `pyproject.toml`
|
||||
- `deploy/torch-constraints.txt`
|
||||
|
||||
```
|
||||
torch==2.9.1
|
||||
torchaudio==2.9.1
|
||||
torchvision==0.24.1
|
||||
```
|
||||
|
||||
Then relock and reinstall:
|
||||
|
||||
```bash
|
||||
uv lock
|
||||
uv sync
|
||||
```
|
||||
|
||||
Confirm the GPU is actually usable before relaunching:
|
||||
|
||||
```bash
|
||||
uv run python -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_capability())"
|
||||
```
|
||||
|
||||
`True (12, 0)` means the kernels are there.
|
||||
|
||||
**Why both files:** `constraint-dependencies` governs `uv sync` / `uv lock` /
|
||||
`uv run`, and `deploy/torch-constraints.txt` governs the `uv pip install`
|
||||
paths (Docker and the Colab notebook), which ignore project-level uv settings.
|
||||
`tests/test_torch_constraints_are_applied.py` fails if the two drift, so a
|
||||
one-sided edit is caught rather than shipped.
|
||||
|
||||
**What you do not have to do:** `torchaudio 2.9` removed `set_audio_backend()`,
|
||||
which VoiceStudio used to call unguarded — that turned this upgrade into a
|
||||
different hard startup crash (`AttributeError` inside `ml_imports`). It is
|
||||
guarded now, so the upgrade path above is clean on a current checkout.
|
||||
|
||||
**Keeping the change:** these are the repo's own pins, so a `git pull` that
|
||||
touches them will conflict or overwrite. Re-apply after updating until the
|
||||
default pin moves — the default cannot move for everyone until the newer torch
|
||||
is verified across the older GPUs VoiceStudio supports, since a build that adds
|
||||
`sm_120` can drop older architectures.
|
||||
|
||||
**Linked issue:** [#1931](https://github.com/debpalash/VoiceStudio/issues/1931)
|
||||
— thanks to the reporter for the full diagnosis, including the verification
|
||||
commands above.
|
||||
|
||||
## 6. `uv venv` Python download fails (restricted network)
|
||||
|
||||
**Symptom:** during first launch, `uv` exits with a network error pulling
|
||||
|
||||
@@ -0,0 +1,91 @@
|
||||
"""`ml_imports` must not call a torchaudio API that may not be there (#1931).
|
||||
|
||||
`torchaudio.set_audio_backend()` was removed in torchaudio 2.9. `soundfile` had
|
||||
been the only backend since 2.0, so the call was already a no-op — but
|
||||
unguarded it raises `AttributeError` inside the `ml_imports` startup phase, and
|
||||
a failure there takes the whole backend down: the desktop app sits on "starting
|
||||
backend" forever and `/health` stays 503.
|
||||
|
||||
That is not hypothetical. RTX 50-series (Blackwell, sm_120) owners have no
|
||||
choice but to move off the pinned torch 2.8.0, which carries no sm_120 kernels,
|
||||
and the torch 2.9.x they land on brings torchaudio 2.9 with it. The one group
|
||||
forced to upgrade met a hard startup crash for a line that does nothing.
|
||||
|
||||
The guard is one `hasattr`. This test is what keeps it: a cleanup pass that
|
||||
sees a no-op call and "simplifies" it by deleting the condition would restore
|
||||
the crash for every Blackwell user, and no test in the suite would notice —
|
||||
CI runs the pinned torch, where the attribute still exists.
|
||||
"""
|
||||
|
||||
import ast
|
||||
from pathlib import Path
|
||||
|
||||
BACKEND = Path(__file__).resolve().parents[1] / "backend"
|
||||
|
||||
# APIs that a supported torchaudio release has already removed. Reading a
|
||||
# missing attribute is an AttributeError, so each of these must be reached only
|
||||
# behind a check that it exists.
|
||||
REMOVED_TORCHAUDIO_APIS = {"set_audio_backend", "get_audio_backend", "list_audio_backends"}
|
||||
|
||||
|
||||
def _guard_names(test: ast.AST) -> set:
|
||||
"""Attribute names an `if` condition proves present.
|
||||
|
||||
Recognises `hasattr(torchaudio, "x")` and `getattr(torchaudio, "x", None)`,
|
||||
including inside a boolean combination, which is how a real guard is
|
||||
written when it also checks something else.
|
||||
"""
|
||||
names = set()
|
||||
for node in ast.walk(test):
|
||||
if not isinstance(node, ast.Call) or not isinstance(node.func, ast.Name):
|
||||
continue
|
||||
if node.func.id not in ("hasattr", "getattr") or len(node.args) < 2:
|
||||
continue
|
||||
target = node.args[1]
|
||||
if isinstance(target, ast.Constant) and isinstance(target.value, str):
|
||||
names.add(target.value)
|
||||
return names
|
||||
|
||||
|
||||
def _unguarded_calls(path: Path) -> list:
|
||||
tree = ast.parse(path.read_text(encoding="utf-8"))
|
||||
parents = {}
|
||||
for parent in ast.walk(tree):
|
||||
for child in ast.iter_child_nodes(parent):
|
||||
parents[child] = parent
|
||||
|
||||
bad = []
|
||||
for node in ast.walk(tree):
|
||||
if not isinstance(node, ast.Attribute) or node.attr not in REMOVED_TORCHAUDIO_APIS:
|
||||
continue
|
||||
if not (isinstance(node.value, ast.Name) and node.value.id == "torchaudio"):
|
||||
continue
|
||||
cursor, guarded = node, False
|
||||
while cursor in parents:
|
||||
cursor = parents[cursor]
|
||||
# Inside a try/except that would catch the AttributeError is also
|
||||
# a legitimate way to survive the removal.
|
||||
if isinstance(cursor, ast.Try):
|
||||
guarded = True
|
||||
break
|
||||
if isinstance(cursor, ast.If) and node.attr in _guard_names(cursor.test):
|
||||
guarded = True
|
||||
break
|
||||
if not guarded:
|
||||
bad.append((node.lineno, node.attr))
|
||||
return bad
|
||||
|
||||
|
||||
def test_no_unguarded_removed_torchaudio_api():
|
||||
offenders = {}
|
||||
for path in sorted(BACKEND.rglob("*.py")):
|
||||
bad = _unguarded_calls(path)
|
||||
if bad:
|
||||
offenders[str(path.relative_to(BACKEND.parent))] = bad
|
||||
assert not offenders, (
|
||||
f"These reach a torchaudio API that torchaudio 2.9 removed, with "
|
||||
f"nothing proving it exists: {offenders}. Wrap it in "
|
||||
'`if hasattr(torchaudio, "..."):` — unguarded it is an AttributeError '
|
||||
"inside ml_imports, which takes the backend down on every RTX "
|
||||
"50-series machine (#1931)."
|
||||
)
|
||||
Reference in New Issue
Block a user