* feat(01-01): encrypted settings store + alembic migration (AUTH-02, T-01-01) Adds the SQLite-backed encrypted settings store that Phase 1 token resolver will read from. Closes the at-rest plaintext risk for HF tokens (T-01-01). - backend/services/settings_store.py: get_hf_token / set_hf_token / clear_hf_token using Fernet symmetric AEAD. Stored value column never contains the literal "hf_" substring. - backend/services/_secret_key.py: per-install Fernet key derived via scrypt(machine-id + 16-byte random salt). machine-id resolution covers macOS (ioreg IOPlatformUUID), Linux (/etc/machine-id and dbus fallback), Windows (HKLM Cryptography MachineGuid via winreg). Final fallback to hostname+user with a warn log. - backend/migrations/versions/0001_phase1_settings_table.py: alembic migration adding `settings(key, value, updated_at)`. Idempotent — checks for an existing table so fresh installs (where _BASE_SCHEMA already created it) and v0.2.7 upgrades both succeed. - backend/core/db.py: _BASE_SCHEMA grows the settings table for fresh installs; init_db() now runs `alembic upgrade head` after the CREATE. - backend/migrations/env.py: honours an externally-set sqlalchemy.url so tests can point alembic at a fixture DB; falls back to core.config DB_PATH for production. - pyproject.toml: cryptography>=41 added explicitly (RESEARCH.md Assumption A1 was checked at execute-time and proved false; the dep was not present transitively, so the install would fail without this). Tests (10 cases, all green): - Round-trip encryption + plaintext-leakage check (T-01-01 invariant) - Salt persistence across clear/set cycles - InvalidToken decrypt path returns None (Open Question #5 resolution) - Concurrent reads consistent under sqlite WAL - Alembic upgrade on a hand-built v0.2.7 fixture DB preserves all existing tables + seeded rows (CLAUDE.md backward-compat constraint) - Alembic downgrade -1 drops only the settings table Refs #35. * feat(01-01): 3-source HF token resolver + log redactor + 5 read sites patched Closes the #35 bug class (bare os.environ.get('HF_TOKEN') reads) by routing every backend HF-token consumer through one resolver, and mitigates T-01-02 (info disclosure via logs) by stripping `hf_[A-Za-z0-9]{30,}` substrings from every log record at the root logger. backend/services/token_resolver.py: - resolve(skip) — 3-source cascade (App → Env → HF-CLI), each source validated via huggingface_hub.whoami(); first valid wins. - on_401(active) — invalidate cache and re-resolve skipping the source that just 401'd (AUTH-06). - state() — three SourceState rows for the Settings UI: set, masked preview (hf_…<last 3>), whoami_user, whoami_ok. - save_app_token / clear_app_token — wraps settings_store + calls huggingface_hub.login(add_to_git_credential=False) per Pitfall #2. - 300-second whoami cache so repeated Settings-page renders don't hit the HF API. backend/core/logging_filter.py: - HFTokenRedactor(logging.Filter) — regex `hf_[A-Za-z0-9]{30,}` so real tokens are masked but `hf_hub` / `hf_token` literals survive. - install_redaction_filter() — idempotent attach to root + every handler. backend/main.py: install the redactor at startup, BEFORE the file handler is added. Re-installed after the file handler attaches so the handler-attached filter list includes it too. Read-side call sites patched (per Pitfall #1 — every HF token read must flow through token_resolver.resolve()): - backend/api/routers/dub_core.py:540 (the original #35 site) - backend/api/routers/system.py:38 (_has_hf_token notification) - backend/services/model_manager.py:480 (diarization pipeline auth) - backend/services/sonitranslate.py:143 (Popen env for SoniTranslate child) - backend/services/sonitranslate.py:217 (gradio_client predict call) New endpoint: - GET /system/hf-token/state — returns the 3-source cascade state with masked tokens for the Wave 2 Settings UI panel. Grep gate confirmed clean: zero `os.environ.get("HF_TOKEN")` reads remain outside token_resolver.py. Tests (17 new cases, all green): - tests/backend/services/test_token_resolver.py: priority cascade, 401 skip mid-resolve, on_401 fallback, state() shape, save+login invariant (add_to_git_credential=False), HUGGING_FACE_HUB_TOKEN alias acceptance. - tests/backend/core/test_logging_filter.py: msg + args redaction, multi-token redaction, non-string args pass-through, short-token literals preserved, install_redaction_filter idempotence. Refs #35. * feat(01-01): Settings hf-token API endpoints + subprocess env injection (AUTH-03/04) Backend half of the Wave 2 Settings → API Keys UI plus the AUTH-04 subprocess env-injection invariant. backend/api/routers/settings.py: - POST /api/settings/hf-token — body {token: str} → save_app_token - DELETE /api/settings/hf-token — also_clear_hf_cli query → clear_app_token - GET /api/settings/hf-token/state — same shape as token_resolver.state() All three are gated by `Depends(require_loopback)` at the router level (threat T-01-03 mitigation; non-loopback Host → 403). backend/main.py: router mounted alongside existing API routers. Subprocess env injection (AUTH-04, threat T-01-04 disposition=accept): - backend/services/sonitranslate.py already updated in Task 2 to read via token_resolver.resolve() and inject HF_TOKEN + YOUR_HF_TOKEN into the SoniTranslate child env block. - backend/services/gpu_sandbox.py: NOT patched — the GPU sandbox runs in-process TTS generation that uses the parent's already-loaded HF state. Adding env injection there is a no-op (parent and child share state via multiprocessing.Pipe before any HF API call). - backend/services/model_manager.py:480 (Task 2): resolves in-process, no subprocess crosses here. - backend/api/routers/exports.py: subprocess.Popen calls only spawn `open` / `explorer` / `xdg-open` — file-manager launchers with no HF needs. Skipped per Task 3 conservative-patching rule. So the canonical AUTH-04 site for this milestone is sonitranslate.py. Future SubprocessBackend work in Phase 2 will inherit the same pattern. Tests (8 new cases, all green): - tests/backend/test_engine_spawn_token.py * POST /hf-token loopback → 200 + state.active == "app" * POST /hf-token non-loopback → 403 ("loopback origin required") * DELETE /hf-token clears settings_store + state.active == None * GET /hf-token/state returns 3 source rows in priority order * GET /hf-token/state non-loopback → 403 * env block contains HF_TOKEN + YOUR_HF_TOKEN when resolver returns one * env block does NOT contain an injected empty HF_TOKEN when resolver returns None * source-level check that backend/services/sonitranslate.py still reads via token_resolver.resolve() (regression guard against silent reverts of the AUTH-04 wiring) Full Wave 1 test suite: 35/35 green. Phase 0 smoke tests still green. Refs #35. * docs(01-01): SUMMARY + STATE update for Phase 1 Wave 1 completion Records execution outcome of the 3-task plan: 10 files created, 9 modified, 35 new test cases, 5 read sites patched, grep gate clean. Documents the two Rule-3/Rule-2 deviations applied (cryptography dep, env.py URL override), the subprocess-launcher inventory for Phase 2, and the known stray edit to the main repo's pyproject.toml that needs a one- line user action to revert. Updates STATE.md current-position table, progress bar, and open TODOs to point at Wave 2 (Plan 01-02) and Wave 3 (Plan 01-03) as the next steps.
231 lines
8.5 KiB
Python
231 lines
8.5 KiB
Python
"""Subprocess env injection + Settings API endpoint tests (Task 3).
|
|
|
|
Covers AUTH-04 (subprocess HF_TOKEN injection) and AUTH-03 backend half
|
|
(Settings API endpoints — POST/DELETE/GET, all loopback-gated).
|
|
"""
|
|
import sys
|
|
from typing import Optional
|
|
from unittest.mock import MagicMock, patch
|
|
|
|
import pytest
|
|
|
|
|
|
SAMPLE_TOKEN = "hf_subsubsubsubsubsubsubsubsubsubsubsubsub01"
|
|
|
|
|
|
@pytest.fixture
|
|
def fresh_app(monkeypatch, tmp_path):
|
|
"""Build a fresh FastAPI app instance with isolated DB + cleared HF env.
|
|
The Settings router is mounted manually because the full main.py app
|
|
factory imports the entire backend stack (torch, whisperx, demucs, …)
|
|
which is too heavy for a unit test."""
|
|
monkeypatch.setenv("OMNIVOICE_DATA_DIR", str(tmp_path))
|
|
monkeypatch.delenv("HF_TOKEN", raising=False)
|
|
monkeypatch.delenv("HUGGING_FACE_HUB_TOKEN", raising=False)
|
|
|
|
for mod in list(sys.modules):
|
|
if (
|
|
mod == "core" or mod.startswith("core.")
|
|
or mod == "services" or mod.startswith("services.")
|
|
or mod == "api" or mod.startswith("api.")
|
|
):
|
|
del sys.modules[mod]
|
|
|
|
from core import db as _db
|
|
_db.init_db()
|
|
|
|
from fastapi import FastAPI
|
|
from api.routers import settings as settings_router
|
|
|
|
app = FastAPI()
|
|
app.include_router(settings_router.router)
|
|
return app
|
|
|
|
|
|
def _client(app):
|
|
"""TestClient anchored to a loopback client tuple so require_loopback
|
|
treats requests as local. The default TestClient client tuple is
|
|
('testclient', 50000), which the dep rejects."""
|
|
from fastapi.testclient import TestClient
|
|
return TestClient(app, client=("127.0.0.1", 12345))
|
|
|
|
|
|
def test_post_hf_token_loopback_succeeds(fresh_app, monkeypatch):
|
|
"""A loopback-origin POST persists the token and returns the updated
|
|
cascade state."""
|
|
import huggingface_hub
|
|
monkeypatch.setattr(huggingface_hub, "login", lambda **kw: None)
|
|
monkeypatch.setattr(
|
|
huggingface_hub,
|
|
"whoami",
|
|
lambda token=None, **kw: {"name": "alice"},
|
|
)
|
|
monkeypatch.setattr(huggingface_hub, "get_token", lambda: None)
|
|
|
|
c = _client(fresh_app)
|
|
r = c.post("/api/settings/hf-token", json={"token": SAMPLE_TOKEN})
|
|
assert r.status_code == 200, r.text
|
|
|
|
# Round-trip via settings_store.
|
|
from services import settings_store
|
|
assert settings_store.get_hf_token() == SAMPLE_TOKEN
|
|
|
|
# Response contains the masked active source.
|
|
body = r.json()
|
|
assert body["active"] == "app"
|
|
assert any(s["source"] == "app" and s["set"] for s in body["sources"])
|
|
|
|
|
|
def test_post_hf_token_non_loopback_returns_403(fresh_app):
|
|
"""A non-loopback origin (simulated via TestClient client tuple) is
|
|
rejected with 403 per the require_loopback dep."""
|
|
from fastapi.testclient import TestClient
|
|
with TestClient(fresh_app, client=("10.0.0.5", 12345)) as c:
|
|
r = c.post("/api/settings/hf-token", json={"token": SAMPLE_TOKEN})
|
|
assert r.status_code == 403
|
|
assert "loopback" in r.json().get("detail", "").lower()
|
|
|
|
|
|
def test_delete_hf_token_clears_store(fresh_app, monkeypatch):
|
|
import huggingface_hub
|
|
monkeypatch.setattr(huggingface_hub, "login", lambda **kw: None)
|
|
monkeypatch.setattr(huggingface_hub, "logout", lambda: None)
|
|
monkeypatch.setattr(
|
|
huggingface_hub,
|
|
"whoami",
|
|
lambda token=None, **kw: {"name": "alice"},
|
|
)
|
|
monkeypatch.setattr(huggingface_hub, "get_token", lambda: None)
|
|
|
|
c = _client(fresh_app)
|
|
c.post("/api/settings/hf-token", json={"token": SAMPLE_TOKEN})
|
|
from services import settings_store
|
|
assert settings_store.get_hf_token() == SAMPLE_TOKEN
|
|
|
|
r = c.delete("/api/settings/hf-token")
|
|
assert r.status_code == 200
|
|
assert settings_store.get_hf_token() is None
|
|
body = r.json()
|
|
assert body["active"] is None
|
|
assert all(not s["set"] for s in body["sources"])
|
|
|
|
|
|
def test_get_hf_token_state_returns_three_rows(fresh_app, monkeypatch):
|
|
"""GET state returns the same shape as token_resolver.state(): three
|
|
SourceState rows in priority order, plus an `active` field."""
|
|
import huggingface_hub
|
|
monkeypatch.setattr(huggingface_hub, "get_token", lambda: None)
|
|
monkeypatch.setattr(
|
|
huggingface_hub,
|
|
"whoami",
|
|
lambda token=None, **kw: {"name": "alice"},
|
|
)
|
|
|
|
c = _client(fresh_app)
|
|
r = c.get("/api/settings/hf-token/state")
|
|
assert r.status_code == 200
|
|
body = r.json()
|
|
assert "active" in body
|
|
assert "sources" in body
|
|
assert [s["source"] for s in body["sources"]] == ["app", "env", "hf-cli"]
|
|
|
|
|
|
def test_get_hf_token_state_loopback_only(fresh_app):
|
|
"""GET state is on the same loopback-only router; non-loopback → 403."""
|
|
from fastapi.testclient import TestClient
|
|
with TestClient(fresh_app, client=("10.0.0.5", 12345)) as c:
|
|
r = c.get("/api/settings/hf-token/state")
|
|
assert r.status_code == 403
|
|
|
|
|
|
def _build_subprocess_env(resolved_token: Optional[str]) -> dict:
|
|
"""Reproduce the env-injection logic that lives in
|
|
backend/services/sonitranslate.py:start(). We extract it here so the
|
|
test exercises the canonical pattern without standing up the whole
|
|
SoniTranslate machinery (which assumes an installed Gradio app on
|
|
disk). The pattern matches the one documented in 01-01-PLAN.md Task 3
|
|
Step 1."""
|
|
import os
|
|
from services import token_resolver
|
|
|
|
env = os.environ.copy()
|
|
resolved = token_resolver.resolve()
|
|
hf_token = resolved.token if resolved else ""
|
|
if hf_token:
|
|
env["HF_TOKEN"] = hf_token
|
|
env["YOUR_HF_TOKEN"] = hf_token
|
|
return env
|
|
|
|
|
|
def test_subprocess_env_includes_hf_token_when_resolved(monkeypatch, tmp_path):
|
|
"""When token_resolver.resolve() returns a token, the env block passed
|
|
to a Popen-style subprocess launcher contains HF_TOKEN=<that token>.
|
|
This is the AUTH-04 invariant; the SoniTranslate launcher (and any
|
|
future subprocess launcher) MUST follow this exact pattern."""
|
|
monkeypatch.setenv("OMNIVOICE_DATA_DIR", str(tmp_path))
|
|
monkeypatch.delenv("HF_TOKEN", raising=False)
|
|
for mod in list(sys.modules):
|
|
if (
|
|
mod == "core" or mod.startswith("core.")
|
|
or mod == "services" or mod.startswith("services.")
|
|
):
|
|
del sys.modules[mod]
|
|
from core import db as _db
|
|
_db.init_db()
|
|
|
|
from services import token_resolver
|
|
fake = token_resolver.ResolvedToken(
|
|
token=SAMPLE_TOKEN, source="app", username="alice"
|
|
)
|
|
monkeypatch.setattr(token_resolver, "resolve", lambda **kw: fake)
|
|
|
|
env = _build_subprocess_env(SAMPLE_TOKEN)
|
|
assert env["HF_TOKEN"] == SAMPLE_TOKEN
|
|
assert env["YOUR_HF_TOKEN"] == SAMPLE_TOKEN
|
|
|
|
|
|
def test_subprocess_env_unchanged_when_no_token(monkeypatch, tmp_path):
|
|
"""When token_resolver.resolve() returns None, the env block does NOT
|
|
contain an injected HF_TOKEN. (If the parent had one in os.environ
|
|
it would still be in the copy — but the launcher does not _add_ an
|
|
empty string, which would clobber any child-set default.)"""
|
|
monkeypatch.setenv("OMNIVOICE_DATA_DIR", str(tmp_path))
|
|
monkeypatch.delenv("HF_TOKEN", raising=False)
|
|
monkeypatch.delenv("HUGGING_FACE_HUB_TOKEN", raising=False)
|
|
for mod in list(sys.modules):
|
|
if (
|
|
mod == "core" or mod.startswith("core.")
|
|
or mod == "services" or mod.startswith("services.")
|
|
):
|
|
del sys.modules[mod]
|
|
from core import db as _db
|
|
_db.init_db()
|
|
|
|
from services import token_resolver
|
|
monkeypatch.setattr(token_resolver, "resolve", lambda **kw: None)
|
|
|
|
env = _build_subprocess_env(None)
|
|
# HF_TOKEN must not have been injected by the launcher.
|
|
assert "HF_TOKEN" not in env or env.get("HF_TOKEN") == ""
|
|
assert "YOUR_HF_TOKEN" not in env
|
|
|
|
|
|
def test_sonitranslate_module_uses_resolver(monkeypatch, tmp_path):
|
|
"""Source-level check: the SoniTranslate module's subprocess.Popen
|
|
launcher block contains the canonical resolver import. This guards
|
|
against future refactors silently reverting the AUTH-04 wiring."""
|
|
import inspect
|
|
monkeypatch.setenv("OMNIVOICE_DATA_DIR", str(tmp_path))
|
|
for mod in list(sys.modules):
|
|
if (
|
|
mod == "core" or mod.startswith("core.")
|
|
or mod == "services" or mod.startswith("services.")
|
|
):
|
|
del sys.modules[mod]
|
|
from services import sonitranslate
|
|
src = inspect.getsource(sonitranslate)
|
|
# The launcher must read from the resolver, not from os.environ.
|
|
assert "token_resolver.resolve" in src
|
|
# And the env injection assigns HF_TOKEN explicitly.
|
|
assert 'env["HF_TOKEN"]' in src or "env['HF_TOKEN']" in src
|