* feat(demos): ship the demo audio and video the app already advertises Every demo asset in the app was a dead link on anything but a Mac. `personalities.py` has carried a `preview_url` for each of the seven voice-design presets since they were added; DictationDemo.jsx posts three bundled WAVs to /transcribe so the feature can be shown without microphone permission; the Dub workspace reads a manifest and plays a source video plus four dubbed languages. None of those files were committed, because the tooling that renders them (scripts/build_demos.sh, scripts/build_dub_demo.sh) hard- requires macOS `say` — it even carries a `TODO: add espeak-ng path for Linux contributors`. So the presets returned 404, the replay buttons did nothing, and the dubbing demo never loaded. Rendered with VoiceStudio's own engine, which runs wherever the app does: - 7 voice-design previews (2.2 MB) - 3 dictation replay clips (1.1 MB) — verified by transcribing them back: the conversational and French clips round-trip exactly - dubbing demo: source + 4 dubbed videos with subtitles and manifest (9.6 MB) Tooling fixes this turned up: - build_dub_demo.sh wrote to backend/assets/demo/dubbing, but main.py mounts backend/assets/samples at /demo_audio — so the frontend's /demo_audio/demo/dubbing/manifest.json could never have resolved even after a successful Mac build. Output moved under the mount. - `say` is now the fallback rather than the requirement: the new scripts/render_dub_demo_audio.py renders the five tracks with the engine and the shell script picks them up. - The five demo paragraphs lived in two files. They are now one JSON both read — two copies is one edit away from a video whose subtitles disagree with it. - render_demos_omnivoice.py peak-normalized, which a single-sample transient defeats: the Helpdesk preset landed at -30 dB RMS against -17 dB for its neighbours, so the preview row played at wildly different volumes. Now EBU R128 at -18 LUFS with a -1.5 dBTP ceiling. - …and pinning the output rate, because loudnorm resamples to 192 kHz internally and writes there unless told otherwise, which turned 2.1 MB of previews into 17.5 MB of identical-sounding audio. - update_manifest() looked for a manifest at a path nothing writes, so it always printed "not found" and did nothing. - Dictation is rendered here now too. It was excluded on the grounds that `say` was good enough and engine TTS was overkill — true only on macOS. tests/test_demo_assets_exist.py resolves every advertised URL against the directory main.py actually mounts, and checks each dubbing subtitle matches the script its manifest entry claims. A missing static file is not an import error and not a failing request; nothing would have caught this otherwise. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs(changelog): stamp the demo-asset entries with their PR ref Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(demos): watermark rendered demo audio, and harden the render scripts Review findings on #1517: - Greptile P1: the renderers wrote engine output straight to disk, so a re-render shipped demo audio with no provenance mark. These clips play back to users as VoiceStudio output — they are synthetic audio leaving the app like any other, and now go through mark_synthetic (#1169), the one chokepoint every producing route uses. It runs on the file AFTER loudnorm, since loudnorm re-encodes what it is handed, and says so loudly when marking is unavailable rather than committing an unmarked asset. The dubbing renderer shares the same helper. - CodeRabbit: build_dub_demo.sh checked only source.src.wav before deciding it could run without macOS `say`, so a Linux or Windows run with four of five tracks present reached a missing one, called `say`, and left a half-built bundle. It now requires all five. - CodeRabbit: shutil.move over an existing path delegates to os.rename, which raises FileExistsError on Windows — os.replace overwrites atomically everywhere. - CodeRabbit: the preview test discovered presets in a parametrize argument, importing app code at collection time and leaving core.personalities in sys.modules for later tests. Discovery moved into the test body. CI: the rendered dub bundle's zh/ja subtitles, its manifest and the script source are dubbing CONTENT, not UI strings — allowlisted in test_no_hardcoded_cjk.py with that justification. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(demos): a render that cannot be watermarked fails instead of warning CodeRabbit and Greptile, #1517: mark_synthetic degrades rather than raising — correct for generation, wrong for a render script, whose whole job is to produce files a human then commits. A printed warning on a scrolling console is not a gate, so both scripts exited 0 with unmarked assets sitting on disk ready to commit. They now raise, with the reason and the fix; OMNIVOICE_DEMO_ALLOW_UNMARKED=1 stays for a local listen. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * ci: stop a flaky dependency fetch from failing green runs en-core-web-sm resolves to a direct GitHub release URL, and github.com intermittently answers `http2 error: refused stream before processing any application logic`. uv's own three retries all land within the same few seconds and fail together, so the whole job dies on a dependency that has nothing to do with the change under test — it cost #1518 and #1517 an otherwise-green run tonight. Two changes: back off between whole `uv sync` attempts, which is what actually clears it, and pass --no-sync to the pytest steps. `uv run` re-resolves the environment before running, so every test step was a fresh chance to hit the same fetch even though the install step had already synced — that is exactly how #1518 failed, in the isolated backend/tests step, with all 5467 tests already passed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * ci: one retry seam for every uv sync, not just the job that failed last en-core-web-sm resolves to a direct GitHub *release* URL rather than a package index, and github.com intermittently answers `http2 error: refused stream before processing any application logic`. uv's own retries all land inside the same ~10 seconds and fail together, so a job dies on a dependency unrelated to the change under test. Tonight that cost four otherwise-green runs across #1515, #1517 and #1518 — and the first fix only covered the Tests job, so the next failure simply moved to Smoke (Linux), which syncs separately. The fetch is per-job, so the fix has to be per-job: scripts/uv-sync-retry.sh backs off between whole attempts (15s, 45s, 90s) and every workflow that syncs now goes through it — ci.yml (tests + the platform matrix), release.yml, security.yml, evals.yml. It still fails loudly after four attempts, so a genuinely broken lockfile is not disguised as a flake. The Tests job also lacked the UV_HTTP_TIMEOUT / UV_HTTP_RETRIES the smoke matrix has always set, which is part of why it was the one that kept dying; it has them now. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(ci): pin the Intel-Mac contract by intent, not by command spelling test_ci_verifies_intel_mac_as_the_documented_remote_only_host asserted the literal line `run: uv sync --extra pockettts`, so routing every sync through scripts/uv-sync-retry.sh read as a broken Intel-Mac contract. The contract it exists to protect is that the pockettts extra installs ONLY on backend_supported legs — which the regex now pins, while leaving how the sync is invoked free to change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * ci: keep every uv run out of the resolver, and bound the retry budget CodeRabbit, #1517: - `uv run` re-resolves before running, so the smoke suite, the worker-artifact tests, the release test run and the eval run were each a fresh chance to hit the flaky direct-URL fetch outside the retry loop. All of them pass --no-sync now; the environment is already synced by the step that owns the retries. security.yml's `uv run --with pip-audit` is deliberately left alone — it layers an ephemeral package rather than running the project's own tests. - The retry count multiplied uv's own budget (UV_HTTP_RETRIES=5 with a 120 s timeout on the smoke matrix). Three attempts and 60 s of total backoff outlast the refusals actually observed while staying well inside the jobs' timeout-minutes. - The Intel-Mac contract test pinned the smoke command literally too, so --no-sync tripped it exactly like the sync line did. Same fix: assert the contract (smoke runs only on backend_supported legs), not its spelling. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
194 lines
7.3 KiB
YAML
194 lines
7.3 KiB
YAML
# Security scanning — runs on every PR, on push to main, and weekly.
|
|
#
|
|
# Complements the CodeRabbit + Greptile app reviews (which fire on PR creation)
|
|
# with deterministic, gating checks:
|
|
# • gitleaks — secret scanning. HARD FAIL: a leaked credential blocks merge.
|
|
# • CodeQL — Python + JS/TS SAST. Results land in the Security tab.
|
|
# • bandit — Python SAST (SARIF → Security tab). Reporting, non-gating.
|
|
# • pip-audit — Python dependency advisories. Reporting, non-gating.
|
|
# • bun audit — frontend dependency advisories. Reporting, non-gating.
|
|
#
|
|
# Only the secret scan gates the PR. Dependency advisories and bandit findings
|
|
# are surfaced as signal (Security tab / job log) rather than blocking every PR
|
|
# on a transitive upstream advisory — consistent with the "no ceremony,
|
|
# continuous-to-main" cadence.
|
|
name: Security
|
|
|
|
on:
|
|
pull_request:
|
|
branches: [main]
|
|
push:
|
|
branches: [main]
|
|
schedule:
|
|
# Mondays 06:00 UTC — catch advisories disclosed since the last PR.
|
|
- cron: "0 6 * * 1"
|
|
workflow_dispatch:
|
|
|
|
env:
|
|
# Match ci.yml: run JS actions on Node 24 (GH removes Node 20 in Sep 2026).
|
|
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true
|
|
|
|
# Least privilege by default; jobs that upload SARIF opt into security-events.
|
|
permissions:
|
|
contents: read
|
|
|
|
# PR branches: a new push cancels the superseded scan (no wasted runners).
|
|
# main: every commit keeps its own group, so nothing is cancelled — a merge
|
|
# train used to leave a permanent red ✗ ("cancelled") on every intermediate
|
|
# commit in the history view even though nothing failed.
|
|
concurrency:
|
|
group: security-${{ github.ref }}-${{ github.ref == 'refs/heads/main' && github.sha || 'branch' }}
|
|
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}
|
|
|
|
jobs:
|
|
# ── Secret scanning (gating) ─────────────────────────────────────────────
|
|
# Full-history scan on push to main; PR-diff scan on pull_request (faster,
|
|
# and the action picks the right mode from the event automatically).
|
|
secrets:
|
|
name: Secret scan (gitleaks)
|
|
runs-on: ubuntu-22.04
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
with:
|
|
# gitleaks needs full history to scan all commits on push events.
|
|
fetch-depth: 0
|
|
# No authed git needed after clone; don't persist GITHUB_TOKEN.
|
|
persist-credentials: false
|
|
- name: gitleaks
|
|
uses: gitleaks/gitleaks-action@v2
|
|
env:
|
|
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
# GITLEAKS_LICENSE is only required for GitHub *organizations*; this is
|
|
# a personal public repo, so the action runs free without it.
|
|
|
|
# ── CodeQL SAST (Python + JS/TS) ─────────────────────────────────────────
|
|
codeql:
|
|
name: CodeQL (${{ matrix.language }})
|
|
runs-on: ubuntu-22.04
|
|
permissions:
|
|
contents: read
|
|
security-events: write
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
language: [python, javascript-typescript]
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
with:
|
|
persist-credentials: false
|
|
|
|
- name: Initialize CodeQL
|
|
uses: github/codeql-action/init@v3
|
|
with:
|
|
languages: ${{ matrix.language }}
|
|
# Both targets are interpreted — no compiled build step needed.
|
|
build-mode: none
|
|
# Scope analysis to shipped product code. The excluded trees never
|
|
# ship in the installer's runtime path and produce the bulk of the
|
|
# note-level + false-positive findings (file-not-closed in eval
|
|
# harnesses, unused alembic migration globals, bind-all in tests,
|
|
# path sinks in the legacy Gradio research UI). Queries live in the
|
|
# inline config so there's a single source of truth next to
|
|
# paths-ignore. paths-ignore is supported here because build-mode is
|
|
# `none` (interpreted analysis).
|
|
config: |
|
|
queries:
|
|
- uses: security-and-quality
|
|
paths-ignore:
|
|
- omnivoice/eval
|
|
- research
|
|
- tests
|
|
- backend/migrations
|
|
- "**/*.test.js"
|
|
- "**/*.test.jsx"
|
|
- "**/*.test.ts"
|
|
- "**/*.test.tsx"
|
|
|
|
- name: Analyze
|
|
uses: github/codeql-action/analyze@v3
|
|
with:
|
|
category: "/language:${{ matrix.language }}"
|
|
|
|
# ── Python SAST (bandit → SARIF) ─────────────────────────────────────────
|
|
bandit:
|
|
name: Python SAST (bandit)
|
|
runs-on: ubuntu-22.04
|
|
permissions:
|
|
contents: read
|
|
security-events: write
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
with:
|
|
persist-credentials: false
|
|
|
|
- name: Setup Python 3.11
|
|
uses: actions/setup-python@v5
|
|
with:
|
|
python-version: "3.11"
|
|
|
|
# -ll: report MEDIUM+ severity only. -ii: MEDIUM+ confidence only.
|
|
# Keeps the SARIF focused on findings worth a human look. The scan step
|
|
# is allowed to "fail" (findings present) without failing the job; the
|
|
# SARIF upload still runs so results reach the Security tab.
|
|
#
|
|
# NOTE: the `sarif` output format lives in the `bandit[sarif]` extra
|
|
# (pulls in sarif-om + jschema-to-python). Plain `bandit` rejects
|
|
# `-f sarif`, so install via the extra spec.
|
|
- name: Run bandit
|
|
continue-on-error: true
|
|
run: |
|
|
pipx run --spec 'bandit[sarif]' bandit -r backend/ -ll -ii -f sarif -o bandit.sarif
|
|
|
|
# continue-on-error: this job is reporting-only. If bandit can't write a
|
|
# SARIF for any reason (no findings dir, pipx hiccup), don't fail the job.
|
|
- name: Upload bandit SARIF
|
|
uses: github/codeql-action/upload-sarif@v3
|
|
if: always()
|
|
continue-on-error: true
|
|
with:
|
|
sarif_file: bandit.sarif
|
|
category: bandit
|
|
|
|
# ── Dependency advisories (reporting) ────────────────────────────────────
|
|
dependencies:
|
|
name: Dependency audit
|
|
runs-on: ubuntu-22.04
|
|
steps:
|
|
- uses: actions/checkout@v4
|
|
with:
|
|
persist-credentials: false
|
|
|
|
- name: Install uv
|
|
uses: astral-sh/setup-uv@v3
|
|
with:
|
|
enable-cache: true
|
|
cache-dependency-glob: "uv.lock"
|
|
|
|
- name: Setup Python 3.11
|
|
uses: actions/setup-python@v5
|
|
with:
|
|
python-version: "3.11"
|
|
|
|
# Audit the resolved Python environment. Non-gating: a transitive
|
|
# advisory with no fix available should not wall off every PR.
|
|
- name: pip-audit (Python)
|
|
continue-on-error: true
|
|
run: |
|
|
bash scripts/uv-sync-retry.sh
|
|
uv run --with pip-audit pip-audit
|
|
|
|
# Pin a floor: `bun audit` was added in bun 1.2.x, so guarantee it exists.
|
|
- name: Setup Bun
|
|
uses: oven-sh/setup-bun@v1
|
|
with:
|
|
bun-version: "1.2"
|
|
|
|
# `bun audit` reports advisories against the frontend lockfile. Non-gating
|
|
# for the same reason; also tolerant of older bun without the subcommand.
|
|
- name: bun audit (frontend)
|
|
continue-on-error: true
|
|
working-directory: frontend
|
|
run: |
|
|
bun install --frozen-lockfile
|
|
bun audit
|