velixio-header-optional-system-stats
12
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Palash DebnathandClaude Fable 5
|
48c9a3b1f8 |
feat(settings): compute-device override (auto / CUDA / ROCm / XPU / MPS / CPU) (#1557)
* feat(settings): compute-device override — auto | CUDA | ROCm | XPU | MPS | CPU Auto-detect stays the default; the override kills the 'auto-detect picked wrong' issue class. Applied at the single choke point (_probe()'s family selection) so routing, get_best_device(), and every badge inherit it. Resolution: OMNIVOICE_DEVICE env > Settings pick (prefs.json) > auto (#981 pattern). An override can steer, never invent hardware: a family the host lacks is noted and ignored; cpu is always honorable. Applies at next backend start (host caps are immutable per process — same restart contract as the rest of the Performance tab, RestartBadge shown). GET/PUT /api/settings/compute-device (admin-gated) reports resolved vs applied so the panel shows restart-required truthfully and disables itself under an env pin instead of pretending. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(changelog): entry for the compute-device override (#1557) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(device-override): harvest — the override reaches CT2 ASR, full i18n, honest edge states - _ctranslate2_cuda_ok() and the ASR sidecar now gate on the probe's family, so a cpu pin (or ROCm host) can never hand CTranslate2 a CUDA device — the override reaches every CT2 loader through one shared gate - override_ignored exposed by the API and shown by the panel (env pin naming a device this machine lacks: auto is in effect, restart won't change it) - all 8 panel strings + 5 device-family labels translated into all 21 locales; failed saves keep their error visible through the re-sync - test isolation: cleanup drops OMNIVOICE_DEVICE before re-probing so no overridden caps leak into later tests; panel tests wait for loaded state - xpu/intel search keywords; oxfmt formatting Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(device-override): round 2 — fail-safe probe fallbacks, complete i18n, combined pin state - a broken capability probe now means CPU everywhere (CT2 gate + ASR sidecar) — never a torch-derived guess that would bypass a cpu pin or re-open #1529 on ROCm; regression test added - env-pinned AND not-detected shows both facts in one subtitle - device_load_failed/perf_save_failed translated into all 21 locales; CJK/th/vi/ar strings no longer say literal 'Auto' - test_ctranslate2_never_gets_cuda_on_a_rocm_build pins the probe family (it was order-dependent on the lru_cache before) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(test): pin the probe family in the faster-whisper OOM-fallback test Same class as the rocm-build test: it mocked torch but not the probe the new override gate consults first, so on a cpu-family CI host the CUDA fallback chain under test was unreachable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Palash DebnathandClaude Fable 5
|
b79ba9bd3b |
docs(readme): lead with download + first clone; seed benchmarks page (#1555)
* docs(readme): lead with download + first clone; seed benchmarks page Quickstart (installers, install guides, a three-step first-clone walkthrough) moves above What's-new/Features in both READMEs — visitors get the action before the pitch. New docs/benchmarks.md anchors measured per-engine/device numbers on the bench_pipeline.py harness, community-contributed, no estimates. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(changelog): entry for the README conversion restructure (#1555) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(bench): emit RTF + CUDA peak VRAM; guard NaN RAM; define the benchmarks schema Bot harvest on #1555: the tts stage now prints RTF per warm measurement and CUDA peak VRAM (None elsewhere — no made-up zeros), the stage floor refuses unmeasurable RAM instead of sailing past a NaN comparison (FLOOR_GB=0 overrides), docs/benchmarks.md columns map 1:1 to what the harness prints, and the download badges say they open the release page. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(readme): link palash.dev from the maker section Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(bench): name the resolved engine, track VRAM from resolution, comment the guards Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(readme): the quick-switch gif is the hero image The hero shows motion now; the Launchpad screenshot moves into the 0.5.0 What's-new slot so nothing appears twice. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(bench): peak VRAM is reserved memory; adapter engines name their model Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(bench): subprocess-isolated engines report VRAM n/a, not a parent-side zero Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(bench): out-of-process detection is declarative; sherpa rows name their model 'runs_out_of_process' is now a TTSBackend attribute set by SubprocessBackend AND omnivoice-gguf (which inherits TTSBackend directly but spawns a binary per generate — the isinstance check missed it). Duck-typed for the same module-purge reason as _is_subprocess_isolated. Sherpa-onnx identity comes from _model_dir's basename when _model_id is absent. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(bench): backends self-report model identity via TTSBackend.model_identity() Greptile enumerated the adapter engines one at a time (mlx _model_id, sherpa _model_dir, cosyvoice env-only) — the attribute sniffing rots per engine. The hook fixes the class: each multi-model backend reports its own identity, the profiler just asks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Palash DebnathandClaude Opus 5
|
41722afe3b |
refactor(launchpad): quieter, borderless design refresh (#1515)
* refactor(launchpad): quieter, borderless design refresh The launchpad carried decoration from an earlier direction: icon chips, corner-hung count badges, a permanently visible filled arrow, uppercase mono card titles, and a dotted stipple divider — plus a frame that had been invisible since the app-wide border tokens were zeroed. Rework it around what the borderless direction actually implies: - Feature tiles get a whisper-faint surface instead of a dead frame, and read as three bands (bare glyph + count / title + arrow / description). `--card-hue` is spent sparingly — the glyph at rest, the surface, count and arrow only once raised. Titles move to sans sentence case; counts are plain tabular numerals. Lift softened 4px -> 2px, coloured glow -> neutral shadow, plus an explicit focus ring and a staggered entrance. - Hero drops the boxed "646" pill and the filled A/B-Compare button for quiet type, with a hairline standing in for the separation. - Section labels trade the dotted stipple for a single fading hairline; rows are transparent until hover and reveal "Open" on hover/focus (it stays in the DOM, so AT and keyboard always reach it). - Hero, tiles, recent files, callout and project lists now share one 1180px column — previously only the top half was capped, so lists ran edge-to-edge on a wide display while the deck stayed centred. Two bugs found and fixed while doing it: - Buttons that had `border border-solid border-transparent` removed fell back to the UA default border and rendered a visible 1px outline. They now carry `border-0` explicitly. - `.lp-animate` used `animation-fill-mode: both`, so after the entrance it kept owning `transform` — and animation-origin declarations outrank normal ones, which silently killed the card hover lift. Now `backwards`, which still holds the from-state through the stagger delay. Also drops CSS the page has not rendered since #904: the cursor-spotlight layer, the breath ring, and the per-card waveform strip. Verified with headless renders at 1600/1280/940 and the empty state. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(dictation): decode Wayland portal signals and show the capture pill The GlobalShortcuts portal declares Activated/Deactivated as (o session, s shortcut_id, t timestamp, a{sv} options). We decoded the timestamp as u32, so zbus rejected every signal with Signature mismatch: got `(osta{sv})`, expected `(osua{sv})` and the press was dropped as an invalid signal. Registration succeeded and the desktop even reported the bound chord back, so the hotkey looked wired up while doing nothing at all — on every Wayland compositor, for the whole life of the feature (#1490). Decode the 64-bit timestamp, and keep the 32-bit spelling as a fallback so a non-conforming portal degrades to working rather than to silence. With presses arriving, the second half of the failure showed: nothing had shown the widget window since it became a hidden recorder host, so a capture ran with no pill on screen — and a mic or Accessibility failure rendered into a window nobody could see. Add show_dictation_pill, which bottom-centres the capsule on the monitor under the pointer and shows it without taking focus (Windows keeps SW_SHOWNOACTIVATE so paste still lands in the user's document), and call it from the widget for every state but idle. Wayland denies clients their own placement, so the compositor picks the spot there; the pill still appears. dispatch_dictation_capture now logs whether a press was emitted or queued — a press that reaches Rust and produces nothing was otherwise indistinguishable from one the compositor never delivered. Tests: portal signals decode at both timestamp widths (the 64-bit case fails before this change with the exact production error); pill placement centres, respects a second monitor's origin, and clamps rather than going off-screen; the widget shows for a state needing the user, stays hidden while idle, and never shows for a press that arrives while dictation is disabled. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: sync in-progress workspace changes Uncommitted work already in the tree, checkpointed so the branch matches the local machine: - Remote GPU workers: join-from-the-app flow, one-time secrets, QR join codes, a Compute control in the status bar, and the device-list Workers panel (#1516) - Model Catalogue workspace, with Settings pointing at it - Settings sidebar search and keyboard navigation - Demo assets for dubbing, dictation and voice design, plus the scripts that render them - Backend: validation-error handling, ASR request-path degradation, and the accompanying tests - CHANGELOG entries for the above and for the Wayland dictation fix Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(tests): follow Engines to the Model Catalogue, and green the sweep - test_supertonic3 asserted the license gate points at "Settings" while the engine now names Model Catalogue → Engines, which is where the accept button actually lives. The assertion follows the move; what it pins is unchanged — the hint must name a place the user can reach it. - Carries the CJK allowlist entries for the rendered dub bundle (#1517) and the regenerated route snapshot for /workers/agent (#1516), both of which this branch inherits from the workspace sync. - docs/install/linux.md: the dictation capsule is bottom-anchored everywhere except Wayland, where the protocol gives applications no say in their placement. Documented rather than left as a surprise (CodeRabbit). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * ci: stop a flaky dependency fetch from failing green runs en-core-web-sm resolves to a direct GitHub release URL, and github.com intermittently answers `http2 error: refused stream before processing any application logic`. uv's own three retries all land within the same few seconds and fail together, so the whole job dies on a dependency that has nothing to do with the change under test — it cost #1518 and #1517 an otherwise-green run tonight. Two changes: back off between whole `uv sync` attempts, which is what actually clears it, and pass --no-sync to the pytest steps. `uv run` re-resolves the environment before running, so every test step was a fresh chance to hit the same fetch even though the install step had already synced — that is exactly how #1518 failed, in the isolated backend/tests step, with all 5467 tests already passed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * ci: one retry seam for every uv sync, not just the job that failed last en-core-web-sm resolves to a direct GitHub *release* URL rather than a package index, and github.com intermittently answers `http2 error: refused stream before processing any application logic`. uv's own retries all land inside the same ~10 seconds and fail together, so a job dies on a dependency unrelated to the change under test. Tonight that cost four otherwise-green runs across #1515, #1517 and #1518 — and the first fix only covered the Tests job, so the next failure simply moved to Smoke (Linux), which syncs separately. The fetch is per-job, so the fix has to be per-job: scripts/uv-sync-retry.sh backs off between whole attempts (15s, 45s, 90s) and every workflow that syncs now goes through it — ci.yml (tests + the platform matrix), release.yml, security.yml, evals.yml. It still fails loudly after four attempts, so a genuinely broken lockfile is not disguised as a flake. The Tests job also lacked the UV_HTTP_TIMEOUT / UV_HTTP_RETRIES the smoke matrix has always set, which is part of why it was the one that kept dying; it has them now. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(ci): pin the Intel-Mac contract by intent, not by command spelling test_ci_verifies_intel_mac_as_the_documented_remote_only_host asserted the literal line `run: uv sync --extra pockettts`, so routing every sync through scripts/uv-sync-retry.sh read as a broken Intel-Mac contract. The contract it exists to protect is that the pockettts extra installs ONLY on backend_supported legs — which the regex now pins, while leaving how the sync is invoked free to change. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * ci: keep every uv run out of the resolver, and bound the retry budget CodeRabbit, #1517: - `uv run` re-resolves before running, so the smoke suite, the worker-artifact tests, the release test run and the eval run were each a fresh chance to hit the flaky direct-URL fetch outside the retry loop. All of them pass --no-sync now; the environment is already synced by the step that owns the retries. security.yml's `uv run --with pip-audit` is deliberately left alone — it layers an ephemeral package rather than running the project's own tests. - The retry count multiplied uv's own budget (UV_HTTP_RETRIES=5 with a 120 s timeout on the smoke matrix). Three attempts and 60 s of total backoff outlast the refusals actually observed while staying well inside the jobs' timeout-minutes. - The Intel-Mac contract test pinned the smoke command literally too, so --no-sync tripped it exactly like the sync line did. Same fix: assert the contract (smoke runs only on backend_supported legs), not its spelling. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Palash Debnath
|
95a35b8e07 |
feat(indextts): add native IndexTTS 2.5 support (#1485)
* feat(indextts): add native 2.5 sidecar support * fix(indextts): preserve legacy language metadata * docs(indextts): state model license terms accurately * fix: preserve IndexTTS upgrades and duration controls * fix: complete IndexTTS upgrade safeguards |
||
|
Palash Debnath
|
bcb547b9f2 |
fix(engines): a slow venv probe is not a broken venv (#1414) (#1421)
Every subprocess engine confirms a candidate interpreter by spawning it and importing the engine package. For IndexTTS that is 'import indextts.infer_v2', which pulls in torch and transformers — seconds with a warm page cache, tens of seconds on a first run, a spinning disk, a network share, or Windows with real-time AV scanning every DLL. The bound was 10s (15s for three peers), and elapsing it was treated as a negative: the candidate was discarded exactly as if the import had raised. A working OMNIVOICE_INDEXTTS_DIR install was reported as 'IndexTTS-2 is not installed', or fell through into the lazy bootstrap and reinstalled over a working clone. Only successful resolution was memoised, so every retry re-ran the probe and failed identically — which is why all three reported repro paths look like one bug. A timeout is the absence of evidence, not evidence of breakage. The probe is now tri-state: yes (imported), no (ran and failed), unproven (did not finish). An unproven candidate is kept as a fallback and used only after every candidate has had its chance, so a wedged user clone cannot shadow a healthy bootstrapped venv. If an unproven venv really is broken it now fails at the sidecar handshake with a real error rather than a confident lie about the install. Fixed as a class: backend/engines/_venv_probe.py replaces the drifted copy in each of the four bootstraps, and the bound is tunable per engine, defaulting to 60s. Zero and negative values are ignored — an unbounded probe would let one wedged candidate hang engine resolution forever. Reported with a precise root cause by @OracleNightmare. (#1414) |
||
|
Palash Debnath
|
5cab8e0149 |
feat: rename the product to VoiceStudio (previously OmniVoice-Studio)
Renames what users see. The app, the installers, the window title, the
docs and all 21 locales now say VoiceStudio, with "(previously
OmniVoice-Studio)" noted near the title of each doc surface so people
recognise it.
Deliberately NOT renamed, because renaming any of them silently breaks
an existing install — there is no legacy-path fallback anywhere in this
codebase:
- bundle identifier com.debpalash.omnivoice-studio (MSI UpgradeCode,
macOS TCC grants, managed venv, WebView localStorage, the
single-instance lock)
- data directories OmniVoice / .omnivoice and omnivoice.db
- the ~150 OMNIVOICE_* environment variables
- the X-OmniVoice-* HTTP headers (a wire protocol)
- the published Docker image paths
- the OmniVoice ENGINE, which is a model name and not this product
tests/test_identity_paths_survive_the_rename.py pins every one of those
so a future well-meaning sweep cannot orphan a user's library.
Linux .deb users install a new package name and should apt remove
omnivoice-studio; that note is in the changelog.
|
||
|
Palash DebnathandClaude Opus 5
|
5afcdf787d |
fix(engines): warn before a long CPU synth burns the whole budget (#1302)
* fix(engines): warn before a long CPU synth burns the whole budget #1288 closed the under-provisioned-GPU gap but left the CPU one open, and I missed it: a CPU-only host is a BENIGN routing verdict, so routingNotice() correctly stays silent — yet #1299 and #1260 are exactly that shape, CPU hosts that hit the 300s budget on long text with no warning at all. "Nothing is misconfigured" and "this will finish in time" are different claims. Threshold is the backend's own definition of past-short: generate_timeout_for() gives the first 1200 characters the flat budget before extending it, so ordinary sentences on a CPU laptop stay quiet and only the shape that actually times out is flagged. Hardware caveats still take precedence — one toast, and it names the real reason rather than generic advice. 5 tests; engines.cpuLongText translated in all 21 locales. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(engines): don't tell CPU-tuned engines to switch to themselves Greptile P1. The advice names OmniVoice GGUF and Supertonic-3 as the CPU-tuned alternatives — shown to someone already running one of them, it is advice to switch to what they are using. Those two now get the same warning without the self-referential clause; the engine set matches the backend's own timeout message so the two can't disagree about who is CPU-tuned. Also documents the preflight in docs/performance.md (docs-sync rule): both warning shapes, why the threshold is 1200 characters (it is the figure the budget itself uses), that they are advisory and once-per-engine-per-session, and the CPU-tuned exception. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
debpalash
|
865be7510f |
Merge remote-tracking branch 'origin/main' into fix/1191-tts-stranded-on-cpu
# Conflicts: # CHANGELOG.md |
||
|
debpalashandClaude Opus 4.8
|
fc7fbf1227 |
fix(gpu-pool): bound execution, not queue wait (#1190, #1202)
A job queued behind a busy 1-worker pool burned its whole 300s budget without executing an instruction, then reported "too heavy for the available compute". The clock now starts when a worker picks the job up; queue wait has its own generous bound and surfaces as a retryable saturation error. Also: reset() no longer cancels innocent queued peers; the timeout message stops claiming capacity was restored (the abandoned job keeps the device until it drains); every GPU dispatch uses the shared length-scaled budget; watermark embeds move off the GPU pool; /v1/audio/speech gets 429/503 + Retry-After; a timed-out batch segment fails the job instead of shipping a silent gap. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
debpalashandClaude Opus 4.8
|
0f70744105 |
fix(tts): stop stranding the TTS model on CPU after a dub abort (#1191)
`offload_tts_for_asr()` moves the TTS model to CPU to make VRAM room for WhisperX, but its partner `restore_tts_after_asr()` was only reachable on the dub-transcribe success path. Any abort, terminal error, or client disconnect skipped it, and `get_model()` never re-checked placement — so EVERY subsequent /generate ran on CPU (10-50x slower, CPU pegged) until the ~15-minute idle unload happened to fire. Reported as "speed varies by time of day"; it is fully deterministic. Two independent guarantees: - Balance the pair at the call site: gen()'s `finally` now pays the restore debt on every exit path, chained off the ASR unload so the two never contend for VRAM (and fire-and-forget, since the finally also runs under GeneratorExit where awaiting is illegal). - Self-heal placement (the class fix): `get_model()` verifies the model is on the resolved target device and moves it back if not, so a future unbalanced offload path cannot strand it either. Cheapest-first probe — one parameter check on the hot path; unified memory is exempt (its offload releases the model rather than moving it). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
debpalashandClaude Fable 5
|
63fd497caf |
feat: TTS-only first run, platform-curated ASR, guided OS permissions, parakeet-mlx
Only the TTS model (~2.4 GB) is required on first run; ASR models are per-platform curated picks (curated_on in models.yaml) installed on demand. Every transcription surface returns a typed asr_model_missing error with a one-click download CTA instead of silently pulling multi-GB Whisper weights. Settings -> Models is a grouped, platform-aware catalog. New guided permissions UX (wizard System Check + Settings -> Permissions + mic pre-flight) with native mic-state checks and OS settings deep-links. New parakeet-mlx engine brings Parakeet TDT v3 to Apple Silicon (language-gated capture preference so multilingual dictation never regresses). Docs: expressive-speech page, Flush/Unload + CPU-fallback triage, clone-length FAQ. Hardening: preflight fails open for custom model pins, ROCm curation no longer inherits NVIDIA picks, Windows mic probe reads the NonPackaged consent key, CaptureWidget setup race fixed, offline-cache CI simulation fixes so empty-cache runners stay green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
|
58c6f37252 |
perf(dub): single-use per-segment refs no longer evict the prompts a dub reuses; add docs/performance.md (#1132)
* perf(dub): single-use per-segment refs no longer evict the prompts a dub reuses; add docs/performance.md The scan-resistance fix: A dub cuts a distinct reference clip per segment (Wave 3.2 / #486 — each line clones its own source delivery) and falls back to the per-speaker clone for segments under 3 s. Both paths flow through the voice-clone prompt cache — an LRU of 8. Streaming hundreds of one-shot per-segment clips through that LRU evicts the per-speaker and locked-profile prompts that every fallback segment reuses, so the speaker ref was re-encoded (~0.4 s each, measured with scripts/bench_pipeline.py) again and again across the render. Note what this deliberately does NOT do: the bench's "166 misses vs 2 speakers" framing suggested keying refs per speaker — but per-segment refs are the intentional prosody-matching feature, and the re-transcription behind them is the #1004 correctness fix. Their encode cost is the price of the feature, not waste. The waste was only the eviction side-effect, and that's what this removes: _get_clone_prompt(store=False) still reads the cache (a hit is free) but never inserts, and the dub loop marks exactly the segment-scoped refs (auto-seg: bindings and auto: bindings resolved to a segment clip) as single-use. Per-speaker, locked-profile, and preview refs cache as before. cache_ref is popped in generate_with_cached_ref before the model call — the model's generate() has an explicit signature and would TypeError — and unknown engines ignore it (**kw adapters). The doc: docs/performance.md is the first performance documentation in the repo — none of the ~15 perf env vars appeared anywhere in docs/, the Performance panel's only control is Windows-only, and slowness reports (#1032) arrived as mysteries instead of settings checks. Covers the three classic causes of "it got slow", where generation/dub time goes, every knob with defaults and warnings (raising OMNIVOICE_GPU_WORKERS on a small GPU is the #567 crash, not a speedup), platform notes, and how to run the bench so reports carry numbers. Linked from README's install section. Tests: store=False semantics (encodes, never inserts, still reads), the flood scenario end to end (a speaker prompt stays warm through 3x the cache cap of one-shots), and the pop contract (cache_ref never reaches the model). Full suite: 2974 passed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs,dub: review round — qualify the per-file cache claim; note the OOM-retry tradeoff - CodeRabbit: docs/performance.md's "the reference encode is cached per file" now carves out the dub's per-line clips (single-use by design — nothing for a cache to save). - Greptile P2 (OOM retry re-encodes a single-use ref): acknowledged in a code comment as deliberate — caching the retry's ref would reintroduce the eviction this flag prevents, to optimize a path that only runs after an OOM already cost seconds. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(performance): probe-based torch.compile wording; honest accelerator + cache claims (review) Greptile's repeated OOM-retry finding is deliberately skipped: retaining the prompt across the retry would require passing prompt objects through the adapter protocol (backend.generate takes paths), to save 0.4s on a path that only runs after an OOM already cost seconds — the tradeoff is documented at the call site. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: mergetest <nizam4103@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |