Files
VoiceStudio/tests/backend/test_engine_spawn_token.py
T
Palash Debnath 4a6b978df9 Phase 1 Wave 1: HF token persistence + redactor (closes #35) (#91)
* feat(01-01): encrypted settings store + alembic migration (AUTH-02, T-01-01)

Adds the SQLite-backed encrypted settings store that Phase 1 token resolver
will read from. Closes the at-rest plaintext risk for HF tokens (T-01-01).

- backend/services/settings_store.py: get_hf_token / set_hf_token /
  clear_hf_token using Fernet symmetric AEAD. Stored value column never
  contains the literal "hf_" substring.
- backend/services/_secret_key.py: per-install Fernet key derived via
  scrypt(machine-id + 16-byte random salt). machine-id resolution covers
  macOS (ioreg IOPlatformUUID), Linux (/etc/machine-id and dbus fallback),
  Windows (HKLM Cryptography MachineGuid via winreg). Final fallback to
  hostname+user with a warn log.
- backend/migrations/versions/0001_phase1_settings_table.py: alembic
  migration adding `settings(key, value, updated_at)`. Idempotent — checks
  for an existing table so fresh installs (where _BASE_SCHEMA already
  created it) and v0.2.7 upgrades both succeed.
- backend/core/db.py: _BASE_SCHEMA grows the settings table for fresh
  installs; init_db() now runs `alembic upgrade head` after the CREATE.
- backend/migrations/env.py: honours an externally-set sqlalchemy.url so
  tests can point alembic at a fixture DB; falls back to core.config
  DB_PATH for production.
- pyproject.toml: cryptography>=41 added explicitly (RESEARCH.md
  Assumption A1 was checked at execute-time and proved false; the dep was
  not present transitively, so the install would fail without this).

Tests (10 cases, all green):
- Round-trip encryption + plaintext-leakage check (T-01-01 invariant)
- Salt persistence across clear/set cycles
- InvalidToken decrypt path returns None (Open Question #5 resolution)
- Concurrent reads consistent under sqlite WAL
- Alembic upgrade on a hand-built v0.2.7 fixture DB preserves all
  existing tables + seeded rows (CLAUDE.md backward-compat constraint)
- Alembic downgrade -1 drops only the settings table

Refs #35.

* feat(01-01): 3-source HF token resolver + log redactor + 5 read sites patched

Closes the #35 bug class (bare os.environ.get('HF_TOKEN') reads) by routing
every backend HF-token consumer through one resolver, and mitigates
T-01-02 (info disclosure via logs) by stripping `hf_[A-Za-z0-9]{30,}`
substrings from every log record at the root logger.

backend/services/token_resolver.py:
  - resolve(skip)   — 3-source cascade (App → Env → HF-CLI), each source
    validated via huggingface_hub.whoami(); first valid wins.
  - on_401(active) — invalidate cache and re-resolve skipping the source
    that just 401'd (AUTH-06).
  - state()        — three SourceState rows for the Settings UI: set,
    masked preview (hf_…<last 3>), whoami_user, whoami_ok.
  - save_app_token / clear_app_token — wraps settings_store + calls
    huggingface_hub.login(add_to_git_credential=False) per Pitfall #2.
  - 300-second whoami cache so repeated Settings-page renders don't hit
    the HF API.

backend/core/logging_filter.py:
  - HFTokenRedactor(logging.Filter) — regex `hf_[A-Za-z0-9]{30,}` so real
    tokens are masked but `hf_hub` / `hf_token` literals survive.
  - install_redaction_filter() — idempotent attach to root + every handler.

backend/main.py: install the redactor at startup, BEFORE the file
handler is added. Re-installed after the file handler attaches so the
handler-attached filter list includes it too.

Read-side call sites patched (per Pitfall #1 — every HF token read must
flow through token_resolver.resolve()):
  - backend/api/routers/dub_core.py:540  (the original #35 site)
  - backend/api/routers/system.py:38     (_has_hf_token notification)
  - backend/services/model_manager.py:480 (diarization pipeline auth)
  - backend/services/sonitranslate.py:143 (Popen env for SoniTranslate child)
  - backend/services/sonitranslate.py:217 (gradio_client predict call)

New endpoint:
  - GET /system/hf-token/state — returns the 3-source cascade state with
    masked tokens for the Wave 2 Settings UI panel.

Grep gate confirmed clean: zero `os.environ.get("HF_TOKEN")` reads remain
outside token_resolver.py.

Tests (17 new cases, all green):
  - tests/backend/services/test_token_resolver.py: priority cascade, 401
    skip mid-resolve, on_401 fallback, state() shape, save+login
    invariant (add_to_git_credential=False), HUGGING_FACE_HUB_TOKEN
    alias acceptance.
  - tests/backend/core/test_logging_filter.py: msg + args redaction,
    multi-token redaction, non-string args pass-through, short-token
    literals preserved, install_redaction_filter idempotence.

Refs #35.

* feat(01-01): Settings hf-token API endpoints + subprocess env injection (AUTH-03/04)

Backend half of the Wave 2 Settings → API Keys UI plus the AUTH-04
subprocess env-injection invariant.

backend/api/routers/settings.py:
  - POST /api/settings/hf-token       — body {token: str} → save_app_token
  - DELETE /api/settings/hf-token     — also_clear_hf_cli query → clear_app_token
  - GET /api/settings/hf-token/state  — same shape as token_resolver.state()
  All three are gated by `Depends(require_loopback)` at the router level
  (threat T-01-03 mitigation; non-loopback Host → 403).

backend/main.py: router mounted alongside existing API routers.

Subprocess env injection (AUTH-04, threat T-01-04 disposition=accept):
  - backend/services/sonitranslate.py already updated in Task 2 to read
    via token_resolver.resolve() and inject HF_TOKEN + YOUR_HF_TOKEN into
    the SoniTranslate child env block.
  - backend/services/gpu_sandbox.py: NOT patched — the GPU sandbox runs
    in-process TTS generation that uses the parent's already-loaded HF
    state. Adding env injection there is a no-op (parent and child share
    state via multiprocessing.Pipe before any HF API call).
  - backend/services/model_manager.py:480 (Task 2): resolves in-process,
    no subprocess crosses here.
  - backend/api/routers/exports.py: subprocess.Popen calls only spawn
    `open` / `explorer` / `xdg-open` — file-manager launchers with no
    HF needs. Skipped per Task 3 conservative-patching rule.

So the canonical AUTH-04 site for this milestone is sonitranslate.py.
Future SubprocessBackend work in Phase 2 will inherit the same pattern.

Tests (8 new cases, all green):
  - tests/backend/test_engine_spawn_token.py
    * POST /hf-token loopback → 200 + state.active == "app"
    * POST /hf-token non-loopback → 403 ("loopback origin required")
    * DELETE /hf-token clears settings_store + state.active == None
    * GET /hf-token/state returns 3 source rows in priority order
    * GET /hf-token/state non-loopback → 403
    * env block contains HF_TOKEN + YOUR_HF_TOKEN when resolver returns one
    * env block does NOT contain an injected empty HF_TOKEN when resolver
      returns None
    * source-level check that backend/services/sonitranslate.py still
      reads via token_resolver.resolve() (regression guard against
      silent reverts of the AUTH-04 wiring)

Full Wave 1 test suite: 35/35 green. Phase 0 smoke tests still green.

Refs #35.

* docs(01-01): SUMMARY + STATE update for Phase 1 Wave 1 completion

Records execution outcome of the 3-task plan: 10 files created, 9 modified,
35 new test cases, 5 read sites patched, grep gate clean. Documents the
two Rule-3/Rule-2 deviations applied (cryptography dep, env.py URL
override), the subprocess-launcher inventory for Phase 2, and the
known stray edit to the main repo's pyproject.toml that needs a one-
line user action to revert.

Updates STATE.md current-position table, progress bar, and open TODOs to
point at Wave 2 (Plan 01-02) and Wave 3 (Plan 01-03) as the next steps.
2026-05-20 05:10:37 +05:30

231 lines
8.5 KiB
Python

"""Subprocess env injection + Settings API endpoint tests (Task 3).
Covers AUTH-04 (subprocess HF_TOKEN injection) and AUTH-03 backend half
(Settings API endpoints — POST/DELETE/GET, all loopback-gated).
"""
import sys
from typing import Optional
from unittest.mock import MagicMock, patch
import pytest
SAMPLE_TOKEN = "hf_subsubsubsubsubsubsubsubsubsubsubsubsub01"
@pytest.fixture
def fresh_app(monkeypatch, tmp_path):
"""Build a fresh FastAPI app instance with isolated DB + cleared HF env.
The Settings router is mounted manually because the full main.py app
factory imports the entire backend stack (torch, whisperx, demucs, …)
which is too heavy for a unit test."""
monkeypatch.setenv("OMNIVOICE_DATA_DIR", str(tmp_path))
monkeypatch.delenv("HF_TOKEN", raising=False)
monkeypatch.delenv("HUGGING_FACE_HUB_TOKEN", raising=False)
for mod in list(sys.modules):
if (
mod == "core" or mod.startswith("core.")
or mod == "services" or mod.startswith("services.")
or mod == "api" or mod.startswith("api.")
):
del sys.modules[mod]
from core import db as _db
_db.init_db()
from fastapi import FastAPI
from api.routers import settings as settings_router
app = FastAPI()
app.include_router(settings_router.router)
return app
def _client(app):
"""TestClient anchored to a loopback client tuple so require_loopback
treats requests as local. The default TestClient client tuple is
('testclient', 50000), which the dep rejects."""
from fastapi.testclient import TestClient
return TestClient(app, client=("127.0.0.1", 12345))
def test_post_hf_token_loopback_succeeds(fresh_app, monkeypatch):
"""A loopback-origin POST persists the token and returns the updated
cascade state."""
import huggingface_hub
monkeypatch.setattr(huggingface_hub, "login", lambda **kw: None)
monkeypatch.setattr(
huggingface_hub,
"whoami",
lambda token=None, **kw: {"name": "alice"},
)
monkeypatch.setattr(huggingface_hub, "get_token", lambda: None)
c = _client(fresh_app)
r = c.post("/api/settings/hf-token", json={"token": SAMPLE_TOKEN})
assert r.status_code == 200, r.text
# Round-trip via settings_store.
from services import settings_store
assert settings_store.get_hf_token() == SAMPLE_TOKEN
# Response contains the masked active source.
body = r.json()
assert body["active"] == "app"
assert any(s["source"] == "app" and s["set"] for s in body["sources"])
def test_post_hf_token_non_loopback_returns_403(fresh_app):
"""A non-loopback origin (simulated via TestClient client tuple) is
rejected with 403 per the require_loopback dep."""
from fastapi.testclient import TestClient
with TestClient(fresh_app, client=("10.0.0.5", 12345)) as c:
r = c.post("/api/settings/hf-token", json={"token": SAMPLE_TOKEN})
assert r.status_code == 403
assert "loopback" in r.json().get("detail", "").lower()
def test_delete_hf_token_clears_store(fresh_app, monkeypatch):
import huggingface_hub
monkeypatch.setattr(huggingface_hub, "login", lambda **kw: None)
monkeypatch.setattr(huggingface_hub, "logout", lambda: None)
monkeypatch.setattr(
huggingface_hub,
"whoami",
lambda token=None, **kw: {"name": "alice"},
)
monkeypatch.setattr(huggingface_hub, "get_token", lambda: None)
c = _client(fresh_app)
c.post("/api/settings/hf-token", json={"token": SAMPLE_TOKEN})
from services import settings_store
assert settings_store.get_hf_token() == SAMPLE_TOKEN
r = c.delete("/api/settings/hf-token")
assert r.status_code == 200
assert settings_store.get_hf_token() is None
body = r.json()
assert body["active"] is None
assert all(not s["set"] for s in body["sources"])
def test_get_hf_token_state_returns_three_rows(fresh_app, monkeypatch):
"""GET state returns the same shape as token_resolver.state(): three
SourceState rows in priority order, plus an `active` field."""
import huggingface_hub
monkeypatch.setattr(huggingface_hub, "get_token", lambda: None)
monkeypatch.setattr(
huggingface_hub,
"whoami",
lambda token=None, **kw: {"name": "alice"},
)
c = _client(fresh_app)
r = c.get("/api/settings/hf-token/state")
assert r.status_code == 200
body = r.json()
assert "active" in body
assert "sources" in body
assert [s["source"] for s in body["sources"]] == ["app", "env", "hf-cli"]
def test_get_hf_token_state_loopback_only(fresh_app):
"""GET state is on the same loopback-only router; non-loopback → 403."""
from fastapi.testclient import TestClient
with TestClient(fresh_app, client=("10.0.0.5", 12345)) as c:
r = c.get("/api/settings/hf-token/state")
assert r.status_code == 403
def _build_subprocess_env(resolved_token: Optional[str]) -> dict:
"""Reproduce the env-injection logic that lives in
backend/services/sonitranslate.py:start(). We extract it here so the
test exercises the canonical pattern without standing up the whole
SoniTranslate machinery (which assumes an installed Gradio app on
disk). The pattern matches the one documented in 01-01-PLAN.md Task 3
Step 1."""
import os
from services import token_resolver
env = os.environ.copy()
resolved = token_resolver.resolve()
hf_token = resolved.token if resolved else ""
if hf_token:
env["HF_TOKEN"] = hf_token
env["YOUR_HF_TOKEN"] = hf_token
return env
def test_subprocess_env_includes_hf_token_when_resolved(monkeypatch, tmp_path):
"""When token_resolver.resolve() returns a token, the env block passed
to a Popen-style subprocess launcher contains HF_TOKEN=<that token>.
This is the AUTH-04 invariant; the SoniTranslate launcher (and any
future subprocess launcher) MUST follow this exact pattern."""
monkeypatch.setenv("OMNIVOICE_DATA_DIR", str(tmp_path))
monkeypatch.delenv("HF_TOKEN", raising=False)
for mod in list(sys.modules):
if (
mod == "core" or mod.startswith("core.")
or mod == "services" or mod.startswith("services.")
):
del sys.modules[mod]
from core import db as _db
_db.init_db()
from services import token_resolver
fake = token_resolver.ResolvedToken(
token=SAMPLE_TOKEN, source="app", username="alice"
)
monkeypatch.setattr(token_resolver, "resolve", lambda **kw: fake)
env = _build_subprocess_env(SAMPLE_TOKEN)
assert env["HF_TOKEN"] == SAMPLE_TOKEN
assert env["YOUR_HF_TOKEN"] == SAMPLE_TOKEN
def test_subprocess_env_unchanged_when_no_token(monkeypatch, tmp_path):
"""When token_resolver.resolve() returns None, the env block does NOT
contain an injected HF_TOKEN. (If the parent had one in os.environ
it would still be in the copy — but the launcher does not _add_ an
empty string, which would clobber any child-set default.)"""
monkeypatch.setenv("OMNIVOICE_DATA_DIR", str(tmp_path))
monkeypatch.delenv("HF_TOKEN", raising=False)
monkeypatch.delenv("HUGGING_FACE_HUB_TOKEN", raising=False)
for mod in list(sys.modules):
if (
mod == "core" or mod.startswith("core.")
or mod == "services" or mod.startswith("services.")
):
del sys.modules[mod]
from core import db as _db
_db.init_db()
from services import token_resolver
monkeypatch.setattr(token_resolver, "resolve", lambda **kw: None)
env = _build_subprocess_env(None)
# HF_TOKEN must not have been injected by the launcher.
assert "HF_TOKEN" not in env or env.get("HF_TOKEN") == ""
assert "YOUR_HF_TOKEN" not in env
def test_sonitranslate_module_uses_resolver(monkeypatch, tmp_path):
"""Source-level check: the SoniTranslate module's subprocess.Popen
launcher block contains the canonical resolver import. This guards
against future refactors silently reverting the AUTH-04 wiring."""
import inspect
monkeypatch.setenv("OMNIVOICE_DATA_DIR", str(tmp_path))
for mod in list(sys.modules):
if (
mod == "core" or mod.startswith("core.")
or mod == "services" or mod.startswith("services.")
):
del sys.modules[mod]
from services import sonitranslate
src = inspect.getsource(sonitranslate)
# The launcher must read from the resolver, not from os.environ.
assert "token_resolver.resolve" in src
# And the env injection assigns HF_TOKEN explicitly.
assert 'env["HF_TOKEN"]' in src or "env['HF_TOKEN']" in src