Palash DebnathandClaude Opus 4.8
24a00bea64
* test(probe): add spec-driven AI-agent test harness (L1/L2/L4/L5 + report + triage) Introduces `tests/probe/`, a portable, mostly-deterministic test harness built on the Actor/Judge split: AI agents may drive and self-heal, but verdicts are always deterministic code + metrics — no LLM on the verdict path. Layers: - L1 API: Schemathesis property-fuzz over in-process ASGI (enable-on-demand). - L2 web: Playwright Driver + deterministic self-heal (id→test-id→text, loosened CSS) → pluggable Healer; LLMHealer/anthropic_healer for genuine agentic heal. Judges + self-heal logic unit-tested offline via FakePage; live browser skips. - L4 media: audio correctness — exists/decode/duration/not-silent/clipping/NaN, round-trip ASR WER (pure-python, faster-whisper backend), speaker similarity. No golden-WAV (device-stable metrics only); naturalness is advisory-only. - L5 env/first-run: fresh-data-dir backend boot in a SUBPROCESS (no session contamination), asserts health + DB init + endpoint reachability. Docker gated. Plus: hybrid YAML spec engine + JudgeResult/registry; self-contained HTML report that auto-opens (suppressed in CI/headless/PROBE_NO_OPEN); Triager that clusters failures and drafts a prefilled GitHub issue URL (sanitized, no auto-submit) with a one-click button in the report. Dependency-light: runs in the base venv; schemathesis/resemblyzer/playwright/ anthropic are enable-on-demand and skip cleanly. Generated reports gitignored. Full suite green (657 passed); no contamination of existing tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(probe): add L3 desktop layer (Tauri config-integrity + guarded launch) Per the architecture decision, desktop E2E is substituted by backend-over-HTTP (L5) + browser (L2) since Tauri has no official macOS WebDriver. L3 guards the packaging/shell contract a browser test can't see, against the real tauri.conf.json (with platform-override merge), running on any platform with no Tauri toolchain: - version parity between tauri.conf.json and pyproject (release integrity) - dev/build wiring (devUrl matches the Vite frontend, frontendDist, before* cmds) - bundled binaries first-run depends on (uv / ffmpeg / ffprobe in externalBin) - CSP actually permits the local backend origins (desktop-only failure mode: packaged app can't reach :3900 while the browser build works) Adds desktop.py (config load + platform deep-merge + bundle discovery + launch guard), judges/desktop.py (config_present/config_eq/config_contains/csp_allows), desktop_smoke.probe.yaml, and tests covering integrity, platform-merge replace semantics, and a live bundle launch that skips without a built bundle/display. Full suite green (662 passed). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
137 lines
7.3 KiB
Plaintext
137 lines
7.3 KiB
Plaintext
# ─────────────────────────────────────────────────────────────────────────
|
|
# Python
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
__pycache__/
|
|
*.pyc
|
|
*.pyo
|
|
*.so
|
|
*.egg
|
|
*.egg-info/
|
|
.venv/
|
|
.pytest_cache/
|
|
.mypy_cache/
|
|
.ruff_cache/
|
|
dist/
|
|
build/
|
|
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
# Node / Turborepo / Tauri
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
node_modules/
|
|
.turbo/
|
|
bun.lockb
|
|
frontend/src-tauri/target/
|
|
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
# Secrets & env
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
.env
|
|
.env.local
|
|
.env.*.local
|
|
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
# OS junk
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
.DS_Store
|
|
Thumbs.db
|
|
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
# Editor / tool caches
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
# Ignore ad-hoc Claude Code state, but allow project-bundled skills
|
|
# (CLAUDE.md invites `.claude/skills/<name>/SKILL.md`).
|
|
.claude/*
|
|
!.claude/skills/
|
|
!.claude/skills/**
|
|
/.cache*
|
|
/.tmp/
|
|
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
# Research clones — upstream repos used as reference, not shipped
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
research/
|
|
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
# Runtime data & artifacts
|
|
# (app-local state — safe to regenerate)
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
omnivoice_data/
|
|
/data/
|
|
/local/
|
|
/results/
|
|
/download
|
|
/run*
|
|
*.db
|
|
*.db-shm
|
|
*.db-wal
|
|
*.log
|
|
*.wav
|
|
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
# Frozen regression fixture (GATE-01) — explicit allow-list.
|
|
# The patterns above (omnivoice_data/, *.db, *.wav) would otherwise hide
|
|
# tests/fixtures/omnivoice_data/ from git. Phase 0 requires this fixture
|
|
# to be checked in (no LFS) so every PR's smoke matrix loads it.
|
|
#
|
|
# We only un-ignore the EXACT files the seed script produces. The backend
|
|
# creates runtime subdirs (dub_jobs/, outputs/, preview/, etc.) when the
|
|
# smoke test boots — those stay ignored.
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
!tests/fixtures/omnivoice_data/
|
|
!tests/fixtures/omnivoice_data/README.md
|
|
!tests/fixtures/omnivoice_data/omnivoice.db
|
|
!tests/fixtures/omnivoice_data/voices/
|
|
!tests/fixtures/omnivoice_data/voices/test-voice/
|
|
!tests/fixtures/omnivoice_data/voices/test-voice/profile.json
|
|
!tests/fixtures/omnivoice_data/voices/test-voice/sample.wav
|
|
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
# Bundled demo assets (synthetic, rendered by scripts/build_demos.sh).
|
|
# Ship with the installer so first-run users have working demos before any
|
|
# model weights download. *.wav above would hide these.
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
!backend/assets/samples/**/*.wav
|
|
!backend/assets/samples/*.wav
|
|
*.jsonl
|
|
demo_recording.webp
|
|
cloudflared
|
|
cloudflared.tgz
|
|
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
# Reference / experiment scratch
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
/exp*/
|
|
example.py
|
|
examples/data*
|
|
examples/download*
|
|
examples/exp*/
|
|
omnivoice.zip
|
|
frontend/src-tauri/binaries/ffmpeg
|
|
|
|
# cuDNN 8 compat libs (auto-installed by scripts/setup_cudnn.py)
|
|
cudnn8_compat/
|
|
test-results/
|
|
|
|
# Research repos (local only)
|
|
research/
|
|
!.planning/research/
|
|
!.planning/research/**
|
|
marketing.md
|
|
.coverage
|
|
/engines/
|
|
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
# Local planning tooling (Spec Kit / GSD) — not part of OmniVoice.
|
|
# NOTE: .claude/skills/omnivoice/ (the MCP skill) IS tracked — only the
|
|
# speckit-* helper skills + .specify config are ignored.
|
|
# ─────────────────────────────────────────────────────────────────────────
|
|
.specify/
|
|
.claude/skills/speckit-*/
|
|
.antigravitycli/
|
|
|
|
playwright-report/
|
|
.last-run.json
|
|
|
|
# probe — generated HTML reports
|
|
tests/probe/reports/
|