* feat(01-01): encrypted settings store + alembic migration (AUTH-02, T-01-01) Adds the SQLite-backed encrypted settings store that Phase 1 token resolver will read from. Closes the at-rest plaintext risk for HF tokens (T-01-01). - backend/services/settings_store.py: get_hf_token / set_hf_token / clear_hf_token using Fernet symmetric AEAD. Stored value column never contains the literal "hf_" substring. - backend/services/_secret_key.py: per-install Fernet key derived via scrypt(machine-id + 16-byte random salt). machine-id resolution covers macOS (ioreg IOPlatformUUID), Linux (/etc/machine-id and dbus fallback), Windows (HKLM Cryptography MachineGuid via winreg). Final fallback to hostname+user with a warn log. - backend/migrations/versions/0001_phase1_settings_table.py: alembic migration adding `settings(key, value, updated_at)`. Idempotent — checks for an existing table so fresh installs (where _BASE_SCHEMA already created it) and v0.2.7 upgrades both succeed. - backend/core/db.py: _BASE_SCHEMA grows the settings table for fresh installs; init_db() now runs `alembic upgrade head` after the CREATE. - backend/migrations/env.py: honours an externally-set sqlalchemy.url so tests can point alembic at a fixture DB; falls back to core.config DB_PATH for production. - pyproject.toml: cryptography>=41 added explicitly (RESEARCH.md Assumption A1 was checked at execute-time and proved false; the dep was not present transitively, so the install would fail without this). Tests (10 cases, all green): - Round-trip encryption + plaintext-leakage check (T-01-01 invariant) - Salt persistence across clear/set cycles - InvalidToken decrypt path returns None (Open Question #5 resolution) - Concurrent reads consistent under sqlite WAL - Alembic upgrade on a hand-built v0.2.7 fixture DB preserves all existing tables + seeded rows (CLAUDE.md backward-compat constraint) - Alembic downgrade -1 drops only the settings table Refs #35. * feat(01-01): 3-source HF token resolver + log redactor + 5 read sites patched Closes the #35 bug class (bare os.environ.get('HF_TOKEN') reads) by routing every backend HF-token consumer through one resolver, and mitigates T-01-02 (info disclosure via logs) by stripping `hf_[A-Za-z0-9]{30,}` substrings from every log record at the root logger. backend/services/token_resolver.py: - resolve(skip) — 3-source cascade (App → Env → HF-CLI), each source validated via huggingface_hub.whoami(); first valid wins. - on_401(active) — invalidate cache and re-resolve skipping the source that just 401'd (AUTH-06). - state() — three SourceState rows for the Settings UI: set, masked preview (hf_…<last 3>), whoami_user, whoami_ok. - save_app_token / clear_app_token — wraps settings_store + calls huggingface_hub.login(add_to_git_credential=False) per Pitfall #2. - 300-second whoami cache so repeated Settings-page renders don't hit the HF API. backend/core/logging_filter.py: - HFTokenRedactor(logging.Filter) — regex `hf_[A-Za-z0-9]{30,}` so real tokens are masked but `hf_hub` / `hf_token` literals survive. - install_redaction_filter() — idempotent attach to root + every handler. backend/main.py: install the redactor at startup, BEFORE the file handler is added. Re-installed after the file handler attaches so the handler-attached filter list includes it too. Read-side call sites patched (per Pitfall #1 — every HF token read must flow through token_resolver.resolve()): - backend/api/routers/dub_core.py:540 (the original #35 site) - backend/api/routers/system.py:38 (_has_hf_token notification) - backend/services/model_manager.py:480 (diarization pipeline auth) - backend/services/sonitranslate.py:143 (Popen env for SoniTranslate child) - backend/services/sonitranslate.py:217 (gradio_client predict call) New endpoint: - GET /system/hf-token/state — returns the 3-source cascade state with masked tokens for the Wave 2 Settings UI panel. Grep gate confirmed clean: zero `os.environ.get("HF_TOKEN")` reads remain outside token_resolver.py. Tests (17 new cases, all green): - tests/backend/services/test_token_resolver.py: priority cascade, 401 skip mid-resolve, on_401 fallback, state() shape, save+login invariant (add_to_git_credential=False), HUGGING_FACE_HUB_TOKEN alias acceptance. - tests/backend/core/test_logging_filter.py: msg + args redaction, multi-token redaction, non-string args pass-through, short-token literals preserved, install_redaction_filter idempotence. Refs #35. * feat(01-01): Settings hf-token API endpoints + subprocess env injection (AUTH-03/04) Backend half of the Wave 2 Settings → API Keys UI plus the AUTH-04 subprocess env-injection invariant. backend/api/routers/settings.py: - POST /api/settings/hf-token — body {token: str} → save_app_token - DELETE /api/settings/hf-token — also_clear_hf_cli query → clear_app_token - GET /api/settings/hf-token/state — same shape as token_resolver.state() All three are gated by `Depends(require_loopback)` at the router level (threat T-01-03 mitigation; non-loopback Host → 403). backend/main.py: router mounted alongside existing API routers. Subprocess env injection (AUTH-04, threat T-01-04 disposition=accept): - backend/services/sonitranslate.py already updated in Task 2 to read via token_resolver.resolve() and inject HF_TOKEN + YOUR_HF_TOKEN into the SoniTranslate child env block. - backend/services/gpu_sandbox.py: NOT patched — the GPU sandbox runs in-process TTS generation that uses the parent's already-loaded HF state. Adding env injection there is a no-op (parent and child share state via multiprocessing.Pipe before any HF API call). - backend/services/model_manager.py:480 (Task 2): resolves in-process, no subprocess crosses here. - backend/api/routers/exports.py: subprocess.Popen calls only spawn `open` / `explorer` / `xdg-open` — file-manager launchers with no HF needs. Skipped per Task 3 conservative-patching rule. So the canonical AUTH-04 site for this milestone is sonitranslate.py. Future SubprocessBackend work in Phase 2 will inherit the same pattern. Tests (8 new cases, all green): - tests/backend/test_engine_spawn_token.py * POST /hf-token loopback → 200 + state.active == "app" * POST /hf-token non-loopback → 403 ("loopback origin required") * DELETE /hf-token clears settings_store + state.active == None * GET /hf-token/state returns 3 source rows in priority order * GET /hf-token/state non-loopback → 403 * env block contains HF_TOKEN + YOUR_HF_TOKEN when resolver returns one * env block does NOT contain an injected empty HF_TOKEN when resolver returns None * source-level check that backend/services/sonitranslate.py still reads via token_resolver.resolve() (regression guard against silent reverts of the AUTH-04 wiring) Full Wave 1 test suite: 35/35 green. Phase 0 smoke tests still green. Refs #35. * docs(01-01): SUMMARY + STATE update for Phase 1 Wave 1 completion Records execution outcome of the 3-task plan: 10 files created, 9 modified, 35 new test cases, 5 read sites patched, grep gate clean. Documents the two Rule-3/Rule-2 deviations applied (cryptography dep, env.py URL override), the subprocess-launcher inventory for Phase 2, and the known stray edit to the main repo's pyproject.toml that needs a one- line user action to revert. Updates STATE.md current-position table, progress bar, and open TODOs to point at Wave 2 (Plan 01-02) and Wave 3 (Plan 01-03) as the next steps.
255 lines
8.7 KiB
Python
255 lines
8.7 KiB
Python
"""Tests for backend/services/token_resolver.py — AUTH-01 / AUTH-03 / AUTH-06.
|
|
|
|
Covers the 3-source cascade (App → Env → HF-CLI), the on_401 fallback,
|
|
the state() shape consumed by the Settings UI, and the save/clear API the
|
|
React panel will call.
|
|
"""
|
|
import sys
|
|
from unittest.mock import MagicMock
|
|
|
|
import pytest
|
|
|
|
|
|
APP_TOKEN = "hf_appapp00000000000000000000000000000000abc"
|
|
ENV_TOKEN = "hf_envenv00000000000000000000000000000000def"
|
|
CLI_TOKEN = "hf_clicli00000000000000000000000000000000ghi"
|
|
|
|
|
|
@pytest.fixture
|
|
def fresh_resolver(monkeypatch, tmp_path):
|
|
"""Fresh import of services.token_resolver + a tmp DB-backed settings
|
|
store. Also clears HF env vars so the env source is empty by default."""
|
|
monkeypatch.setenv("OMNIVOICE_DATA_DIR", str(tmp_path))
|
|
monkeypatch.delenv("HF_TOKEN", raising=False)
|
|
monkeypatch.delenv("HUGGING_FACE_HUB_TOKEN", raising=False)
|
|
|
|
# Aggressive purge — see note in test_settings_store.py.
|
|
for mod in list(sys.modules):
|
|
if (
|
|
mod == "core" or mod.startswith("core.")
|
|
or mod == "services" or mod.startswith("services.")
|
|
):
|
|
del sys.modules[mod]
|
|
from core import db as _db
|
|
_db.init_db()
|
|
from services import token_resolver
|
|
token_resolver.invalidate_cache()
|
|
return token_resolver
|
|
|
|
|
|
def _mock_whoami(monkeypatch, mapping):
|
|
"""Patch huggingface_hub.whoami so that a known {token: result} mapping
|
|
drives validity. `mapping[token]` may be a dict like {"name": "alice"}
|
|
or an Exception instance to raise."""
|
|
import huggingface_hub
|
|
|
|
def _fake_whoami(token=None, **kwargs):
|
|
if token in mapping:
|
|
outcome = mapping[token]
|
|
if isinstance(outcome, Exception):
|
|
raise outcome
|
|
return outcome
|
|
raise RuntimeError(f"unexpected token in whoami: {token!r}")
|
|
|
|
monkeypatch.setattr(huggingface_hub, "whoami", _fake_whoami)
|
|
|
|
|
|
def _mock_get_token(monkeypatch, value):
|
|
import huggingface_hub
|
|
monkeypatch.setattr(huggingface_hub, "get_token", lambda: value)
|
|
|
|
|
|
def test_resolve_priority_app_wins(fresh_resolver, monkeypatch):
|
|
"""All three sources set + valid → App wins."""
|
|
tr = fresh_resolver
|
|
from services import settings_store
|
|
settings_store.set_hf_token(APP_TOKEN)
|
|
monkeypatch.setenv("HF_TOKEN", ENV_TOKEN)
|
|
_mock_get_token(monkeypatch, CLI_TOKEN)
|
|
_mock_whoami(monkeypatch, {
|
|
APP_TOKEN: {"name": "alice"},
|
|
ENV_TOKEN: {"name": "bob"},
|
|
CLI_TOKEN: {"name": "carol"},
|
|
})
|
|
tr.invalidate_cache()
|
|
result = tr.resolve()
|
|
assert result is not None
|
|
assert result.token == APP_TOKEN
|
|
assert result.source == "app"
|
|
assert result.username == "alice"
|
|
|
|
|
|
def test_resolve_skips_empty_falls_to_env(fresh_resolver, monkeypatch):
|
|
"""No app token, env set, no CLI → Env wins."""
|
|
tr = fresh_resolver
|
|
monkeypatch.setenv("HF_TOKEN", ENV_TOKEN)
|
|
_mock_get_token(monkeypatch, None)
|
|
_mock_whoami(monkeypatch, {ENV_TOKEN: {"name": "bob"}})
|
|
tr.invalidate_cache()
|
|
result = tr.resolve()
|
|
assert result is not None
|
|
assert result.token == ENV_TOKEN
|
|
assert result.source == "env"
|
|
|
|
|
|
def test_resolve_returns_none_when_all_empty(fresh_resolver, monkeypatch):
|
|
tr = fresh_resolver
|
|
_mock_get_token(monkeypatch, None)
|
|
_mock_whoami(monkeypatch, {})
|
|
tr.invalidate_cache()
|
|
assert tr.resolve() is None
|
|
|
|
|
|
def test_resolve_skips_401_app_falls_to_env(fresh_resolver, monkeypatch):
|
|
"""App set but 401, env set and valid → resolver returns Env, not App.
|
|
AUTH-06 mid-resolve fallback."""
|
|
tr = fresh_resolver
|
|
from services import settings_store
|
|
from huggingface_hub.errors import HfHubHTTPError
|
|
|
|
settings_store.set_hf_token(APP_TOKEN)
|
|
monkeypatch.setenv("HF_TOKEN", ENV_TOKEN)
|
|
_mock_get_token(monkeypatch, None)
|
|
err = HfHubHTTPError("401 Unauthorized", response=MagicMock(status_code=401))
|
|
_mock_whoami(monkeypatch, {
|
|
APP_TOKEN: err,
|
|
ENV_TOKEN: {"name": "bob"},
|
|
})
|
|
tr.invalidate_cache()
|
|
result = tr.resolve()
|
|
assert result is not None
|
|
assert result.source == "env"
|
|
assert result.token == ENV_TOKEN
|
|
|
|
|
|
def test_on_401_cascade(fresh_resolver, monkeypatch):
|
|
"""Active source = 'app', call on_401('app') → next valid source."""
|
|
tr = fresh_resolver
|
|
from services import settings_store
|
|
settings_store.set_hf_token(APP_TOKEN)
|
|
monkeypatch.setenv("HF_TOKEN", ENV_TOKEN)
|
|
_mock_get_token(monkeypatch, None)
|
|
_mock_whoami(monkeypatch, {
|
|
APP_TOKEN: {"name": "alice"},
|
|
ENV_TOKEN: {"name": "bob"},
|
|
})
|
|
tr.invalidate_cache()
|
|
# Initial resolve picks app.
|
|
assert tr.resolve().source == "app"
|
|
# When the consumer reports a 401 from "app", fall back.
|
|
fallback = tr.on_401("app")
|
|
assert fallback is not None
|
|
assert fallback.source == "env"
|
|
|
|
|
|
def test_state_returns_three_rows(fresh_resolver, monkeypatch):
|
|
"""state() returns one row per source in priority order, with masked
|
|
token + whoami fields populated. The Settings UI consumes this."""
|
|
tr = fresh_resolver
|
|
from services import settings_store
|
|
settings_store.set_hf_token(APP_TOKEN)
|
|
monkeypatch.setenv("HF_TOKEN", ENV_TOKEN)
|
|
_mock_get_token(monkeypatch, None)
|
|
_mock_whoami(monkeypatch, {
|
|
APP_TOKEN: {"name": "alice"},
|
|
ENV_TOKEN: {"name": "bob"},
|
|
})
|
|
tr.invalidate_cache()
|
|
s = tr.state()
|
|
assert "sources" in s
|
|
assert "active" in s
|
|
assert [r.source for r in s["sources"]] == ["app", "env", "hf-cli"]
|
|
assert s["active"] == "app"
|
|
# masked format: hf_…<last 3 chars of the token>
|
|
app_row = s["sources"][0]
|
|
assert app_row.set is True
|
|
assert app_row.masked is not None and app_row.masked.startswith("hf_")
|
|
assert app_row.masked.endswith(APP_TOKEN[-3:])
|
|
assert app_row.whoami_user == "alice"
|
|
assert app_row.whoami_ok is True
|
|
# HF-CLI row: unset
|
|
cli_row = s["sources"][2]
|
|
assert cli_row.set is False
|
|
assert cli_row.masked is None
|
|
|
|
|
|
def test_save_writes_both_store_and_login(fresh_resolver, monkeypatch):
|
|
"""save_app_token() must update settings_store AND call
|
|
huggingface_hub.login(token=..., add_to_git_credential=False) so the
|
|
canonical HF file (~/.cache/huggingface/token) is in sync."""
|
|
tr = fresh_resolver
|
|
import huggingface_hub
|
|
|
|
login_calls = []
|
|
monkeypatch.setattr(
|
|
huggingface_hub,
|
|
"login",
|
|
lambda **kw: login_calls.append(kw),
|
|
)
|
|
tr.save_app_token(APP_TOKEN)
|
|
|
|
# Settings store now has it.
|
|
from services import settings_store
|
|
assert settings_store.get_hf_token() == APP_TOKEN
|
|
|
|
# huggingface_hub.login was called with the right kwargs.
|
|
assert len(login_calls) == 1
|
|
call = login_calls[0]
|
|
assert call.get("token") == APP_TOKEN
|
|
# Pitfall #2: must always pass add_to_git_credential=False.
|
|
assert call.get("add_to_git_credential") is False
|
|
|
|
|
|
def test_save_uses_add_to_git_credential_false(fresh_resolver, monkeypatch):
|
|
"""Standalone check for Pitfall #2 (explicit so reviewers see the
|
|
invariant being enforced)."""
|
|
tr = fresh_resolver
|
|
import huggingface_hub
|
|
captured = {}
|
|
def _fake_login(**kwargs):
|
|
captured.update(kwargs)
|
|
monkeypatch.setattr(huggingface_hub, "login", _fake_login)
|
|
tr.save_app_token(APP_TOKEN)
|
|
assert captured.get("add_to_git_credential") is False
|
|
|
|
|
|
def test_clear_app_token_removes_from_store(fresh_resolver, monkeypatch):
|
|
tr = fresh_resolver
|
|
import huggingface_hub
|
|
monkeypatch.setattr(huggingface_hub, "login", lambda **kw: None)
|
|
monkeypatch.setattr(huggingface_hub, "logout", lambda: None)
|
|
|
|
tr.save_app_token(APP_TOKEN)
|
|
from services import settings_store
|
|
assert settings_store.get_hf_token() == APP_TOKEN
|
|
|
|
tr.clear_app_token(also_clear_hf_cli=False)
|
|
assert settings_store.get_hf_token() is None
|
|
|
|
|
|
def test_clear_app_token_also_logs_out_when_requested(fresh_resolver, monkeypatch):
|
|
tr = fresh_resolver
|
|
import huggingface_hub
|
|
monkeypatch.setattr(huggingface_hub, "login", lambda **kw: None)
|
|
logout_calls = []
|
|
monkeypatch.setattr(huggingface_hub, "logout", lambda: logout_calls.append(True))
|
|
|
|
tr.save_app_token(APP_TOKEN)
|
|
tr.clear_app_token(also_clear_hf_cli=True)
|
|
assert logout_calls == [True]
|
|
|
|
|
|
def test_resolve_accepts_hugging_face_hub_token_alias(fresh_resolver, monkeypatch):
|
|
"""HF docs list HUGGING_FACE_HUB_TOKEN as the legacy env-var name; the
|
|
resolver must accept either."""
|
|
tr = fresh_resolver
|
|
monkeypatch.delenv("HF_TOKEN", raising=False)
|
|
monkeypatch.setenv("HUGGING_FACE_HUB_TOKEN", ENV_TOKEN)
|
|
_mock_get_token(monkeypatch, None)
|
|
_mock_whoami(monkeypatch, {ENV_TOKEN: {"name": "bob"}})
|
|
tr.invalidate_cache()
|
|
result = tr.resolve()
|
|
assert result is not None
|
|
assert result.source == "env"
|
|
assert result.token == ENV_TOKEN
|