fix(audio): remove hidden reverb from the mastering pre-stage — reverb is preset-declared only (#986)
* fix(audio): remove hidden reverb from the mastering pre-stage — reverb is preset-declared only (#TBD) Field report (Discord): baked-in echo/reverb on some voices. apply_mastering() hardcoded a Reverb that ran on every non-raw synthesis before the user's preset chain — broadcast shipped reverb it never declared, podcast broke its "no reverb" promise, cinematic/warm got doubled reverb. The mastering pre-stage is now data-driven (MASTERING_CHAIN: highpass + compressor, same params as before) and reverb-free; cinematic/warm keep their user-chosen reverb. Regression tests pin the contract, incl. a burst-then- silence echo-tail check and pedalboard-missing passthrough. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(changelog): hidden mastering reverb entry (#986) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: mergetest <test@local> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Fable 5
mergetest
parent
087309259b
commit
7f77f4d7bf
@@ -11,6 +11,7 @@ The bundled TTS model package (`pyproject.toml`) is versioned independently.
|
||||
### Fixed
|
||||
|
||||
- **First-run no longer dead-ends behind restricted networks (e.g. China).** The system check probed hardcoded huggingface.co, and any failure locked the Continue button — users behind the Great Firewall were stuck on the very first screen, even when they had already configured a working mirror. The check now probes the Hugging Face endpoint actually in effect, an unreachable endpoint is a warning instead of a blocker (models already on disk keep working offline), and when huggingface.co is blocked but the hf-mirror.com community mirror answers, the wizard says so and offers a one-click mirror switch right on the check screen — no restart needed. (#984)
|
||||
- **Voices no longer ship with a hidden echo.** Every non-raw synthesis was getting a small room reverb baked in by the mastering pre-stage — on top of whatever effect preset you chose, so even "Podcast" (which promises *no reverb*) had some, and Cinematic/Warm got it twice. A field report ("a lot of echo/reverb on some of the voices") led straight to it. The mastering stage is now highpass + compressor only; reverb happens only when a preset explicitly declares it. Also documented: cloned voices reproduce the reference clip's room acoustics — dry, close-mic references clone cleanest. (#986)
|
||||
|
||||
## [0.3.11] — 2026-07-05
|
||||
|
||||
|
||||
@@ -105,8 +105,8 @@ def _apply_effect_chain(audio_out, sample_rate, effect_preset, *, skip_mastering
|
||||
|
||||
``skip_mastering`` honors a backend's ``applies_own_mastering`` flag
|
||||
(issue #312): studio engines (e.g. VoxCPM2's native 48 kHz output)
|
||||
opt out of the broadcast Compressor + Reverb chain that's tuned for
|
||||
OmniVoice's 24 kHz clone output. Loudness normalization still runs —
|
||||
opt out of the broadcast highpass + Compressor pre-stage that's tuned
|
||||
for OmniVoice's 24 kHz clone output. Loudness normalization still runs —
|
||||
it's a benign peak scale. Mirrors ``_run_tts`` in openai_compat.py.
|
||||
"""
|
||||
from services.audio_dsp import (
|
||||
|
||||
@@ -237,10 +237,10 @@ def _run_tts(backend, text: str, kw: dict):
|
||||
sr = backend.sample_rate
|
||||
# Engines that already emit mastered, studio-grade audio (e.g. VoxCPM2's
|
||||
# native 48 kHz) opt out of apply_mastering via `applies_own_mastering`.
|
||||
# That chain's Compressor + 8% Reverb is tuned for OmniVoice's 24 kHz clone
|
||||
# output; applied to a studio engine it adds an audible level pump and a
|
||||
# reverb tail that degrade the very output we want clean. Loudness
|
||||
# normalisation still runs — it's a benign peak scale, not dynamics.
|
||||
# That chain's highpass + Compressor is tuned for OmniVoice's 24 kHz clone
|
||||
# output; applied to a studio engine it adds an audible level pump that
|
||||
# degrades the very output we want clean. Loudness normalisation still
|
||||
# runs — it's a benign peak scale, not dynamics.
|
||||
if not getattr(backend, "applies_own_mastering", False):
|
||||
wav = apply_mastering(wav, sample_rate=sr)
|
||||
wav = normalize_audio(wav, target_dBFS=-2.0)
|
||||
|
||||
@@ -1,9 +1,12 @@
|
||||
"""
|
||||
Audio DSP pipeline — broadcast-grade mastering + configurable effects chain.
|
||||
|
||||
The default `apply_mastering()` is the same chain shipped since v0.1.0
|
||||
(highpass + compressor + light reverb). The new `apply_effects_chain()`
|
||||
lets callers build custom pipelines from a list of named effects.
|
||||
`apply_mastering()` is the shared pre-stage that runs before the user's
|
||||
effect preset: highpass + gentle compression only (see `MASTERING_CHAIN`).
|
||||
Reverb is deliberately NOT part of it — it is preset-declared only (e.g.
|
||||
cinematic, warm); a hidden reverb here used to bake echo into every non-raw
|
||||
synthesis, which field reports flagged. `apply_effects_chain()` lets callers
|
||||
build custom pipelines from a list of named effects.
|
||||
|
||||
All effects use Spotify's `pedalboard` library. When pedalboard isn't
|
||||
installed, every function degrades gracefully (returns audio unmodified).
|
||||
@@ -97,24 +100,26 @@ def get_effect_chain(preset_id: str) -> list[dict]:
|
||||
|
||||
# ── Core DSP functions ──────────────────────────────────────────────────
|
||||
|
||||
#: Shared pre-preset mastering stage: highpass + gentle compression ONLY.
|
||||
#: Reverb must never live here — a hidden Reverb in this chain baked echo
|
||||
#: into every non-raw synthesis regardless of the chosen preset (field
|
||||
#: reports of echoey voices; the podcast preset even promises "no reverb").
|
||||
#: Reverb is preset-declared only (see EFFECT_PRESETS: cinematic, warm).
|
||||
MASTERING_CHAIN = [
|
||||
{"type": "highpass", "cutoff_hz": 60},
|
||||
{"type": "compressor", "threshold_db": -15, "ratio": 1.5, "attack_ms": 2.0, "release_ms": 100},
|
||||
]
|
||||
|
||||
|
||||
def apply_mastering(audio_tensor, sample_rate=24000):
|
||||
"""Applies professional Broadcast-grade DSP (EQ, Compressor, light Reverb) to the clone voice."""
|
||||
"""Applies the broadcast pre-stage (highpass + gentle compression) to the clone voice.
|
||||
|
||||
Reverb is intentionally absent — only user-chosen effect presets declare
|
||||
it. Degrades gracefully: pedalboard missing or any DSP error returns the
|
||||
input unmodified.
|
||||
"""
|
||||
try:
|
||||
from pedalboard import Pedalboard, Compressor, Reverb, HighpassFilter
|
||||
import numpy as np
|
||||
board = Pedalboard([
|
||||
HighpassFilter(cutoff_frequency_hz=60),
|
||||
Compressor(threshold_db=-15, ratio=1.5, attack_ms=2.0, release_ms=100),
|
||||
Reverb(room_size=0.10, wet_level=0.08, dry_level=0.95)
|
||||
])
|
||||
audio_np = audio_tensor.cpu().numpy()
|
||||
if audio_np.ndim == 1:
|
||||
audio_np = audio_np[np.newaxis, :]
|
||||
effected = board(audio_np, sample_rate, reset=False)
|
||||
return torch.from_numpy(effected).to(audio_tensor.device)
|
||||
except ImportError:
|
||||
return audio_tensor # Fail gracefully if pedalboard isn't installed
|
||||
return apply_effects_chain(audio_tensor, sample_rate, MASTERING_CHAIN)
|
||||
except Exception as e:
|
||||
logger.warning("Mastering DSP Error: %s", e)
|
||||
return audio_tensor
|
||||
|
||||
@@ -143,9 +143,9 @@ class TTSBackend(ABC):
|
||||
supports_voice_design: bool = False
|
||||
|
||||
#: Whether this engine already emits mastered, studio-grade audio and should
|
||||
#: therefore skip the shared apply_mastering() chain (Compressor + Reverb,
|
||||
#: therefore skip the shared apply_mastering() chain (highpass + Compressor,
|
||||
#: tuned for OmniVoice's 24 kHz output). Studio engines like VoxCPM2 (native
|
||||
#: 48 kHz) set this True so their clean output isn't pumped/reverbed. Loudness
|
||||
#: 48 kHz) set this True so their clean output isn't pumped. Loudness
|
||||
#: normalisation is applied regardless — it's a benign peak scale.
|
||||
applies_own_mastering: bool = False
|
||||
|
||||
|
||||
@@ -56,6 +56,8 @@ Priority: `duration` > `speed`.
|
||||
| `preprocess_prompt` | bool | True | Whether to apply preprocessing to the voice-clone prompt audio (remove long silences in reference audio, add punctuation in the end of reference text). |
|
||||
| `postprocess_output` | bool | True | Apply post-processing to generated audio (remove long silences). |
|
||||
|
||||
> **Tip — reference-clip quality transfers.** Zero-shot cloning mirrors the acoustics of the reference clip, not just the voice: a clip recorded in an echoey room clones echoey. Record dry and close-mic for clean output. No effect preset adds reverb unless you choose one that declares it (Cinematic, Warm).
|
||||
|
||||
## Long-Form Generation
|
||||
|
||||
To support stable long-form speech generation with low VRAM consumption, the text is automatically split into smaller segments when the estimated duration of the generated speech exceeds `audio_chunk_duration`, with each segment producing approximately `audio_chunk_duration` seconds of audio. This approach allows the model to accept arbitrarily long text and generate arbitrarily long speech with near-constant VRAM consumption.
|
||||
|
||||
@@ -6,7 +6,7 @@ Today the longform renderer (Audiobook + Stories) applies a **single-pass** `lou
|
||||
|
||||
Upgrade to **two-pass** `loudnorm`: a first **measure** pass (`print_format=json`, output to `-f null -`) parses the clip's `input_i / input_tp / input_lra / input_thresh / target_offset`, then a second **apply** pass feeds those measured values back as `measured_*` + `offset` + `linear=true`. This lands the output accurately on the preset target. The change is a **runner enhancement** layered over the existing pure builders — the pure `build_loudnorm_filter()` and `LOUDNESS_PRESETS` stay; we add a measure-filter builder, a measured-apply-filter builder, a JSON parser, a measure-cmd argv builder, and an async two-pass orchestrator that runs in `_render_longform_sse` (`backend/api/routers/audiobook.py:345`) between the chapter renders and the final mux. Loudness stays **opt-in** (`loudness: None` default on both `AudiobookRequest` `:151` and `LongformRenderRequest` `:510`), so default cross-platform behavior is unchanged.
|
||||
|
||||
> **Naming note (grounded):** "mastering" already exists in this codebase as `services.audio_dsp.apply_mastering()` (`backend/services/audio_dsp.py:101`) — a per-clip pedalboard EQ/Compressor/Reverb chain used by `/generate`, `/dub`, batch, and stream paths. That is a **different** operation and **is not called** in the longform path (`_render_longform_sse` muxes chapter WAVs straight from `synthesize_chapter`, no `apply_mastering`). The two-pass loudnorm here is the *only* loudness operation in the longform renderer. To avoid conflating the two, the new SSE event is named `"mastering"` deliberately as the user-facing loudness step for longform; this is harmless because the longform stream never emits anything else by that name, but reviewers should know the term is overloaded across the repo.
|
||||
> **Naming note (grounded):** "mastering" already exists in this codebase as `services.audio_dsp.apply_mastering()` (`backend/services/audio_dsp.py:101`) — a per-clip pedalboard highpass/Compressor chain used by `/generate`, `/dub`, batch, and stream paths. That is a **different** operation and **is not called** in the longform path (`_render_longform_sse` muxes chapter WAVs straight from `synthesize_chapter`, no `apply_mastering`). The two-pass loudnorm here is the *only* loudness operation in the longform renderer. To avoid conflating the two, the new SSE event is named `"mastering"` deliberately as the user-facing loudness step for longform; this is harmless because the longform stream never emits anything else by that name, but reviewers should know the term is overloaded across the repo.
|
||||
|
||||
## Problem
|
||||
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
"""
|
||||
Regression tests: the mastering pre-stage must not hide a reverb.
|
||||
|
||||
A hardcoded Reverb inside apply_mastering() used to bake echo into every
|
||||
non-raw synthesis regardless of the chosen effect preset (field reports of
|
||||
echoey voices; the podcast preset even promises "no reverb", and cinematic/
|
||||
warm got doubled reverb). Reverb is preset-declared only — these tests pin
|
||||
that contract.
|
||||
|
||||
Kept separate from test_effects_chain.py on purpose: that file skips
|
||||
entirely without pedalboard, while the data-shape guards here must always
|
||||
run. sys.path for backend imports is handled by tests/conftest.py.
|
||||
"""
|
||||
|
||||
import builtins
|
||||
import math
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
import torch
|
||||
|
||||
from services.audio_dsp import (
|
||||
EFFECT_PRESETS,
|
||||
MASTERING_CHAIN,
|
||||
apply_mastering,
|
||||
)
|
||||
|
||||
|
||||
def _stage_types(chain):
|
||||
return [fx["type"] for fx in chain]
|
||||
|
||||
|
||||
def _make_test_audio(duration_s=1.0, sample_rate=24000) -> torch.Tensor:
|
||||
"""Create a test audio tensor with a simple sine wave."""
|
||||
t = torch.linspace(0, duration_s, int(duration_s * sample_rate))
|
||||
return torch.sin(2 * math.pi * 440 * t).unsqueeze(0) # 440 Hz sine, mono
|
||||
|
||||
|
||||
class TestMasteringChainHasNoHiddenReverb:
|
||||
def test_mastering_chain_contains_no_reverb(self):
|
||||
"""The recurrence guard: nobody re-adds a reverb outside the preset system."""
|
||||
assert "reverb" not in _stage_types(MASTERING_CHAIN)
|
||||
|
||||
def test_mastering_chain_keeps_highpass_and_compressor(self):
|
||||
"""Removing the reverb must not gut the rest of the pre-stage."""
|
||||
types = _stage_types(MASTERING_CHAIN)
|
||||
assert "highpass" in types
|
||||
assert "compressor" in types
|
||||
|
||||
|
||||
class TestPresetReverbContract:
|
||||
@pytest.mark.parametrize("preset_id", ["broadcast", "podcast"])
|
||||
def test_no_reverb_presets_stay_reverb_free(self, preset_id):
|
||||
"""podcast's description literally promises "no reverb"."""
|
||||
assert "reverb" not in _stage_types(EFFECT_PRESETS[preset_id]["chain"])
|
||||
|
||||
@pytest.mark.parametrize("preset_id", ["cinematic", "warm"])
|
||||
def test_user_chosen_reverb_survives(self, preset_id):
|
||||
"""Presets that deliberately declare reverb must keep it."""
|
||||
assert "reverb" in _stage_types(EFFECT_PRESETS[preset_id]["chain"])
|
||||
|
||||
|
||||
class TestApplyMasteringFunctional:
|
||||
def test_returns_same_shape_and_device(self):
|
||||
pytest.importorskip("pedalboard")
|
||||
audio = _make_test_audio()
|
||||
result = apply_mastering(audio, sample_rate=24000)
|
||||
assert isinstance(result, torch.Tensor)
|
||||
assert result.shape == audio.shape
|
||||
assert result.device == audio.device
|
||||
|
||||
def test_no_echo_tail_bleeds_into_silence(self):
|
||||
"""A burst followed by silence must stay silent after mastering.
|
||||
|
||||
Fails with the old hidden Reverb (its tail rings past the burst);
|
||||
passes with highpass + compressor only.
|
||||
"""
|
||||
pytest.importorskip("pedalboard")
|
||||
sr = 24000
|
||||
burst = _make_test_audio(duration_s=0.25, sample_rate=sr)
|
||||
audio = torch.cat([burst, torch.zeros(1, sr)], dim=1) # + 1 s silence
|
||||
result = apply_mastering(audio, sample_rate=sr)
|
||||
# Skip 50 ms after the burst so the filters settle; a reverb tail is
|
||||
# far louder and longer than that.
|
||||
tail = result[:, burst.shape[1] + int(0.05 * sr):]
|
||||
assert tail.abs().max().item() < 1e-3
|
||||
|
||||
def test_passthrough_when_pedalboard_missing(self, monkeypatch):
|
||||
"""Graceful degradation: no pedalboard, audio returned unmodified."""
|
||||
real_import = builtins.__import__
|
||||
|
||||
def no_pedalboard(name, *args, **kwargs):
|
||||
if name == "pedalboard" or name.startswith("pedalboard."):
|
||||
raise ImportError("pedalboard unavailable (simulated)")
|
||||
return real_import(name, *args, **kwargs)
|
||||
|
||||
monkeypatch.setattr(builtins, "__import__", no_pedalboard)
|
||||
monkeypatch.delitem(sys.modules, "pedalboard", raising=False)
|
||||
audio = _make_test_audio()
|
||||
result = apply_mastering(audio, sample_rate=24000)
|
||||
assert torch.equal(result, audio)
|
||||
Reference in New Issue
Block a user