upstream-archive/VoiceStudio
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(cloning): bound exhaustive reference selection

Browse Source
This commit is contained in:
debpalash
2026-08-20 09:30:16 +05:30
parent e9e4d95d06
commit 835a889326
31 changed files with 170 additions and 120 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+1 -1
View File
@@ -31,7 +31,7 @@ the frozen-backend fallback mirror it for their toolchains.
- The OmniVoice guide now covers combining style attributes with a reference clip (consistent instruct stabilizes cloning; the reference wins conflicts), inline pronunciation control (pinyin / CMU phonemes), and corrects the claim that the default engine can't do voice design — it can, from attributes (#1565)
### Fixed
- Minute-long cloning references can no longer exhaust system memory: transcript-free clips are bounded before transcription, while an overlong clip with a supplied transcript asks you to trim both to the same passage (#1578) — thanks @ACKAPOB!
- Cloning references can no longer exhaust system memory: transcript-free clips up to 75 seconds are searched in five bounded passages, longer clips ask to be trimmed, and supplied transcripts remain capped at 20 seconds to preserve alignment (#1578) — thanks @ACKAPOB!
- Stored artifact subpaths now resolve after moving a data directory between Windows, macOS, Linux, and Docker, while traversal and symlink escapes remain blocked (#1559) — thanks @Eman-Yousaf!
- A remote browser hitting an API-key-configured server's admin 403 now gets the API-key login form instead of endless console 403s, while desktop and PIN-only/no-key servers keep the plain loopback error so guests are never offered a login no key can satisfy (#1568) — thanks @paoloantinori!
- The crash-isolated ASR sidecar and its download preflight now agree on which model to load — setting the shared faster-whisper model variable applies to both variants instead of the sidecar quietly using a different one (#1556)
docs/engines/omnivoice.md
+4 -1
View File
@@ -52,7 +52,10 @@ The env var overrides the persisted UI choice.
has none, VoiceStudio transcribes the clip automatically on first use and
saves the result to the profile. A clip with a supplied transcript is limited
to 20 seconds so the two stay aligned; trim both to the same passage. Without
a transcript, longer clips are safely trimmed before automatic transcription.
a transcript, VoiceStudio can search up to 75 seconds in five contiguous,
bounded transcription passes and selects the passage with detected speech.
Longer clips must be trimmed first. If no spoken words are detected, trim to
a clear 3–10 second passage or provide its matching transcript.
- Encoded voice references persist on disk (`prompt_cache/` in the app data
dir), so the first generation with a known voice after a restart skips the
re-encode and any transcription pass. Set `OMNIVOICE_PROMPT_DISK_CACHE=0`
frontend/src/hooks/useTTS.js
+2 -1
View File
@@ -6,6 +6,7 @@ import { playBlobAudio, playPing } from '../utils/media';
import {
StreamingPreviewError,
resolveRemoteTtsTarget,
shouldFallbackToClassic,
streamGenerateSpeech,
supportsStreamingPreview,
} from '../utils/streamingTts';
@@ -318,7 +319,7 @@ export default function useTTS({ selectedProfile, setSelectedProfile, loadHistor
// same error. Surface the backend's actionable message instead.
// Non-retryable failures (transport drop, Web Audio glitch) keep the
// classic fallback: there the whole-file path genuinely can succeed.
if (err.retryable) {
if (!shouldFallbackToClassic(err)) {
addBreadcrumb('generate:stream-retryable-abort');
throw err;
}
frontend/src/i18n/locales/ar.json
+2 -1
View File
@@ -2059,7 +2059,8 @@
"ignored_unsupported": "تم تجاهل التعليمات غير المدعومة: {{items}}",
"ignored_duplicate": "تم التجاهل (تم تعيين الفئة بالفعل): {{items}}",
"ref_audio_unusable": "تعذّر سماع أي كلام في الصوت المرجعي — المقطع صامت أو شبه صامت، فلا يوجد صوت لاستنساخه. سجّل مجددًا أقرب إلى الميكروفون (وتأكد من اختيار جهاز الإدخال الصحيح)، أو اختر مقطعًا آخر.",
"ref_audio_too_long": "يجب ألا يتجاوز المرجع المزود بنص 20 ثانية. قصّ الصوت والنص إلى المقطع نفسه بطول 3–10 ثوانٍ، أو احذف النص ليقص VoiceStudio المقطع ويفرغه تلقائيًا."
"ref_audio_too_long": "يمكن أن يبلغ طول المرجع 20 ثانية كحد أقصى مع نص، أو 75 ثانية من دونه. قصّه إلى مقطع كلام واضح بطول 3–10 ثوانٍ؛ وإذا أرفقت نصًا فقصّه إلى المقطع نفسه.",
"ref_audio_no_speech": "لم يكتشف التعرف التلقائي أي كلمات منطوقة في المرجع. قصّه إلى مقطع كلام واضح بطول 3–10 ثوانٍ أو أرفق نصًا مطابقًا."
},
"sharing": {
"title": "المشاركة والوصول عن بعد",
frontend/src/i18n/locales/de.json
+2 -1
View File
@@ -2059,7 +2059,8 @@
"ignored_unsupported": "Nicht unterstützte Anweisung ignoriert: {{items}}",
"ignored_duplicate": "Ignoriert (Kategorie bereits festgelegt): {{items}}",
"ref_audio_unusable": "Im Referenz-Audio war keine Sprache zu hören — der Clip ist still oder fast still, es gibt also keine Stimme zum Klonen. Nimm näher am Mikrofon neu auf (prüfe, ob das richtige Eingabegerät gewählt ist und der Pegel ausschlägt) oder wähle einen anderen Clip.",
"ref_audio_too_long": "Eine Referenz mit angegebenem Transkript darf höchstens 20 Sekunden lang sein. Kürze Audio und Transkript auf dieselbe 3–10-sekündige Passage oder entferne das Transkript, damit VoiceStudio den Clip automatisch kürzt und transkribiert."
"ref_audio_too_long": "Eine Referenz darf mit Transkript höchstens 20 Sekunden, ohne Transkript höchstens 75 Sekunden lang sein. Kürze sie auf eine klare Sprachpassage von 3–10 Sekunden; bei mitgeliefertem Text muss das Transkript denselben Abschnitt abdecken.",
"ref_audio_no_speech": "Die automatische Spracherkennung hat keine gesprochenen Wörter gefunden. Kürze die Referenz auf eine klare Sprachpassage von 3–10 Sekunden oder gib ein passendes Transkript an."
},
"sharing": {
"title": "Teilen und Fernzugriff",
frontend/src/i18n/locales/en.json
+2 -1
View File
@@ -2730,7 +2730,8 @@
"ignored_unsupported": "Ignored unsupported instruct: {{items}}",
"ignored_duplicate": "Ignored (category already set): {{items}}",
"ref_audio_unusable": "No speech could be heard in the reference audio — the clip is silent or nearly silent, so there is no voice to clone. Record again closer to the microphone (check that the right input device is selected and its level moves while you speak), or choose a different clip.",
"ref_audio_too_long": "A reference with a supplied transcript can be at most 20 seconds. Trim both the audio and transcript to the same 3–10 second passage, or remove the transcript so VoiceStudio can trim and transcribe the clip automatically."
"ref_audio_too_long": "A reference can be at most 20 seconds with a supplied transcript, or 75 seconds without one. Trim it to a clear 3–10 second speech passage; if you supply text, trim the transcript to the same passage.",
"ref_audio_no_speech": "Automatic speech detection found no spoken words in the reference. Trim it to a clear 3–10 second speech passage, or supply a matching transcript."
},
"tts": {
"routingFallback": "Running on CPU — this engine has no GPU path on your machine, so generation will be slower. {{reason}}",
frontend/src/i18n/locales/es.json
+2 -1
View File
@@ -2059,7 +2059,8 @@
"ignored_unsupported": "Instrucción no admitida ignorada: {{items}}",
"ignored_duplicate": "Ignorado (categoría ya establecida): {{items}}",
"ref_audio_unusable": "No se detectó voz en el audio de referencia: el clip está en silencio o casi en silencio, así que no hay voz que clonar. Graba de nuevo más cerca del micrófono (comprueba que está seleccionado el dispositivo de entrada correcto y que el nivel se mueve) o elige otro clip.",
"ref_audio_too_long": "Una referencia con transcripción no puede superar los 20 segundos. Recorta el audio y la transcripción al mismo fragmento de 3–10 segundos, o elimina la transcripción para que VoiceStudio recorte y transcriba el clip automáticamente."
"ref_audio_too_long": "Una referencia puede durar como máximo 20 segundos con transcripción o 75 segundos sin ella. Recórtala a un fragmento de voz claro de 3–10 segundos; si aportas texto, recorta la transcripción al mismo fragmento.",
"ref_audio_no_speech": "La detección automática no encontró palabras habladas en la referencia. Recórtala a un fragmento de voz claro de 3–10 segundos o proporciona una transcripción coincidente."
},
"sharing": {
"title": "Compartir y acceso remoto",
frontend/src/i18n/locales/fr.json
+2 -1
View File
@@ -2059,7 +2059,8 @@
"ignored_unsupported": "Instruction non prise en charge ignorée : {{items}}",
"ignored_duplicate": "Ignoré (catégorie déjà définie) : {{items}}",
"ref_audio_unusable": "Aucune voix n'a été détectée dans l'audio de référence — le clip est silencieux ou presque, il n'y a donc aucune voix à cloner. Réenregistrez plus près du micro (vérifiez que le bon périphérique d'entrée est sélectionné et que le niveau bouge) ou choisissez un autre clip.",
"ref_audio_too_long": "Une référence accompagnée d'une transcription ne peut pas dépasser 20 secondes. Coupez l'audio et la transcription sur le même passage de 3 à 10 secondes, ou retirez la transcription pour laisser VoiceStudio couper et transcrire automatiquement le clip."
"ref_audio_too_long": "Une référence peut durer au maximum 20 secondes avec transcription, ou 75 secondes sans transcription. Coupez-la sur un passage vocal clair de 3 à 10 secondes ; si vous fournissez du texte, coupez la transcription sur le même passage.",
"ref_audio_no_speech": "La détection automatique n'a trouvé aucune parole dans la référence. Coupez-la sur un passage vocal clair de 3 à 10 secondes ou fournissez une transcription correspondante."
},
"sharing": {
"title": "Partage et accès à distance",
frontend/src/i18n/locales/hi.json
+2 -1
View File
@@ -2059,7 +2059,8 @@
"ignored_unsupported": "अनदेखा असमर्थित निर्देश: {{items}}",
"ignored_duplicate": "अनदेखा (श्रेणी पहले से ही सेट): {{items}}",
"ref_audio_unusable": "संदर्भ ऑडियो में कोई आवाज़ सुनाई नहीं दी — क्लिप मौन या लगभग मौन है, इसलिए क्लोन करने के लिए कोई आवाज़ नहीं है। माइक्रोफ़ोन के और पास से दोबारा रिकॉर्ड करें (जांचें कि सही इनपुट डिवाइस चुना गया है) या कोई दूसरी क्लिप चुनें।",
"ref_audio_too_long": "दिए गए ट्रांसक्रिप्ट वाला संदर्भ अधिकतम 20 सेकंड का हो सकता है। ऑडियो और ट्रांसक्रिप्ट दोनों को उसी 3–10 सेकंड के हिस्से तक काटें, या ट्रांसक्रिप्ट हटाएँ ताकि VoiceStudio क्लिप को अपने आप काटकर लिख सके।"
"ref_audio_too_long": "ट्रांसक्रिप्ट के साथ संदर्भ अधिकतम 20 सेकंड और उसके बिना अधिकतम 75 सेकंड का हो सकता है। इसे 3–10 सेकंड के स्पष्ट भाषण अंश तक काटें; टेक्स्ट देने पर ट्रांसक्रिप्ट को भी उसी अंश तक काटें।",
"ref_audio_no_speech": "स्वचालित पहचान को संदर्भ में बोले गए शब्द नहीं मिले। इसे 3–10 सेकंड के स्पष्ट भाषण अंश तक काटें या मेल खाता ट्रांसक्रिप्ट दें।"
},
"sharing": {
"title": "साझाकरण और रिमोट एक्सेस",
frontend/src/i18n/locales/id.json
+2 -1
View File
@@ -2059,7 +2059,8 @@
"ignored_unsupported": "Mengabaikan instruksi yang tidak didukung: {{items}}",
"ignored_duplicate": "Diabaikan (kategori sudah ditetapkan): {{items}}",
"ref_audio_unusable": "Tidak ada suara yang terdengar di audio referensi — klipnya senyap atau hampir senyap, jadi tidak ada suara untuk dikloning. Rekam ulang lebih dekat ke mikrofon (pastikan perangkat input yang benar dipilih dan meterannya bergerak), atau pilih klip lain.",
"ref_audio_too_long": "Referensi dengan transkrip yang disediakan maksimal 20 detik. Potong audio dan transkrip ke bagian 3–10 detik yang sama, atau hapus transkrip agar VoiceStudio memotong dan mentranskripsikan klip secara otomatis."
"ref_audio_too_long": "Referensi maksimal 20 detik dengan transkrip atau 75 detik tanpa transkrip. Potong menjadi bagian ucapan jelas sepanjang 3–10 detik; jika menyertakan teks, potong transkrip ke bagian yang sama.",
"ref_audio_no_speech": "Deteksi otomatis tidak menemukan kata yang diucapkan dalam referensi. Potong menjadi bagian ucapan jelas sepanjang 3–10 detik atau sertakan transkrip yang cocok."
},
"sharing": {
"title": "Berbagi & Akses Jarak Jauh",
frontend/src/i18n/locales/it.json
+2 -1
View File
@@ -2059,7 +2059,8 @@
"ignored_unsupported": "Istruzione non supportata ignorata: {{items}}",
"ignored_duplicate": "Ignorato (categoria già impostata): {{items}}",
"ref_audio_unusable": "Nessuna voce rilevata nell'audio di riferimento: la clip è silenziosa o quasi, quindi non c'è alcuna voce da clonare. Registra di nuovo più vicino al microfono (verifica che sia selezionato il dispositivo di ingresso corretto e che il livello si muova) oppure scegli un'altra clip.",
"ref_audio_too_long": "Un riferimento con trascrizione fornita può durare al massimo 20 secondi. Taglia audio e trascrizione sullo stesso passaggio di 3–10 secondi, oppure rimuovi la trascrizione affinché VoiceStudio tagli e trascriva automaticamente la clip."
"ref_audio_too_long": "Un riferimento può durare al massimo 20 secondi con trascrizione o 75 secondi senza. Taglialo a un passaggio parlato chiaro di 3–10 secondi; se fornisci il testo, taglia la trascrizione sullo stesso passaggio.",
"ref_audio_no_speech": "Il rilevamento automatico non ha trovato parole pronunciate nel riferimento. Taglialo a un passaggio parlato chiaro di 3–10 secondi o fornisci una trascrizione corrispondente."
},
"sharing": {
"title": "Condivisione e accesso remoto",
frontend/src/i18n/locales/ja.json
+2 -1
View File
@@ -2059,7 +2059,8 @@
"ignored_unsupported": "サポートされていない命令は無視されました: {{items}}",
"ignored_duplicate": "無視 (カテゴリはすでに設定されています): {{items}}",
"ref_audio_unusable": "リファレンス音声から声を検出できませんでした。クリップが無音またはほぼ無音のため、クローンする声がありません。マイクに近づいて録音し直すか（正しい入力デバイスが選択されているか確認してください）、別のクリップを選んでください。",
"ref_audio_too_long": "文字起こし付きの参照音声は20秒以内にしてください。音声と文字起こしを同じ3〜10秒の区間に切り詰めるか、文字起こしを削除してVoiceStudioに自動で切り詰めと文字起こしをさせてください。"
"ref_audio_too_long": "参照音声は文字起こし付きなら最長20秒、なしなら最長75秒です。明瞭な発話を含む3〜10秒に切り詰め、テキストを指定する場合は文字起こしも同じ区間に合わせてください。",
"ref_audio_no_speech": "自動音声検出で参照音声から発話が見つかりませんでした。明瞭な発話を含む3〜10秒に切り詰めるか、一致する文字起こしを指定してください。"
},
"sharing": {
"title": "共有とリモートアクセス",
frontend/src/i18n/locales/ko.json
+2 -1
View File
@@ -2059,7 +2059,8 @@
"ignored_unsupported": "지원되지 않는 지침을 무시했습니다: {{items}}",
"ignored_duplicate": "무시됨(카테고리가 이미 설정됨): {{items}}",
"ref_audio_unusable": "레퍼런스 오디오에서 음성이 감지되지 않았습니다. 클립이 무음이거나 거의 무음이라 복제할 목소리가 없습니다. 마이크에 더 가까이서 다시 녹음하거나(올바른 입력 장치가 선택되어 있는지 확인) 다른 클립을 선택하세요.",
"ref_audio_too_long": "텍스트가 제공된 참조 오디오는 최대 20초까지 사용할 수 있습니다. 오디오와 텍스트를 동일한 3~10초 구간으로 자르거나, 텍스트를 삭제하여 VoiceStudio가 자동으로 자르고 받아쓰게 하세요."
"ref_audio_too_long": "참조 오디오는 텍스트가 있으면 최대 20초, 없으면 최대 75초까지 사용할 수 있습니다. 명확한 음성이 있는 3~10초 구간으로 자르고, 텍스트를 제공한다면 같은 구간에 맞추세요.",
"ref_audio_no_speech": "자동 음성 감지에서 참조 오디오의 발화 단어를 찾지 못했습니다. 명확한 음성이 있는 3~10초 구간으로 자르거나 일치하는 텍스트를 제공하세요."
},
"sharing": {
"title": "공유 및 원격 액세스",
frontend/src/i18n/locales/nl.json
+2 -1
View File
@@ -2059,7 +2059,8 @@
"ignored_unsupported": "Genegeerde niet-ondersteunde instructie: {{items}}",
"ignored_duplicate": "Genegeerd (categorie al ingesteld): {{items}}",
"ref_audio_unusable": "Er is geen spraak gehoord in de referentie-audio — de clip is stil of bijna stil, dus er is geen stem om te klonen. Neem opnieuw op dichter bij de microfoon (controleer of het juiste invoerapparaat is geselecteerd en het niveau beweegt) of kies een andere clip.",
"ref_audio_too_long": "Een referentie met een aangeleverd transcript mag maximaal 20 seconden duren. Knip audio en transcript tot dezelfde passage van 3–10 seconden, of verwijder het transcript zodat VoiceStudio de clip automatisch kan inkorten en transcriberen."
"ref_audio_too_long": "Een referentie mag maximaal 20 seconden duren met transcript, of 75 seconden zonder transcript. Knip deze tot een duidelijke gesproken passage van 3–10 seconden; knip een meegeleverd transcript tot dezelfde passage.",
"ref_audio_no_speech": "De automatische spraakdetectie vond geen gesproken woorden in de referentie. Knip deze tot een duidelijke gesproken passage van 3–10 seconden of geef een passend transcript op."
},
"sharing": {
"title": "Delen en externe toegang",
frontend/src/i18n/locales/pl.json
+2 -1
View File
@@ -2059,7 +2059,8 @@
"ignored_unsupported": "Ignorowana nieobsługiwana instrukcja: {{items}}",
"ignored_duplicate": "Ignorowane (kategoria już ustawiona): {{items}}",
"ref_audio_unusable": "W nagraniu referencyjnym nie wykryto mowy — klip jest cichy lub prawie cichy, więc nie ma głosu do sklonowania. Nagraj ponownie bliżej mikrofonu (sprawdź, czy wybrano właściwe urządzenie wejściowe i czy poziom się porusza) albo wybierz inny klip.",
"ref_audio_too_long": "Nagranie referencyjne z podaną transkrypcją może mieć najwyżej 20 sekund. Przytnij dźwięk i transkrypcję do tego samego fragmentu 3–10 sekund albo usuń transkrypcję, aby VoiceStudio automatycznie przyciął i przepisał klip."
"ref_audio_too_long": "Nagranie referencyjne może mieć najwyżej 20 sekund z transkrypcją lub 75 sekund bez niej. Przytnij je do wyraźnego fragmentu mowy 3–10 sekund; jeśli podajesz tekst, przytnij transkrypcję do tego samego fragmentu.",
"ref_audio_no_speech": "Automatyczne wykrywanie mowy nie znalazło wypowiedzianych słów. Przytnij nagranie do wyraźnego fragmentu mowy 3–10 sekund lub podaj pasującą transkrypcję."
},
"sharing": {
"title": "Udostępnianie i dostęp zdalny",
frontend/src/i18n/locales/pt.json
+2 -1
View File
@@ -2059,7 +2059,8 @@
"ignored_unsupported": "Instrução não suportada ignorada: {{items}}",
"ignored_duplicate": "Ignorado (categoria já definida): {{items}}",
"ref_audio_unusable": "Nenhuma fala foi detectada no áudio de referência — o clipe está em silêncio ou quase, então não há voz para clonar. Grave novamente mais perto do microfone (confira se o dispositivo de entrada correto está selecionado e se o nível se move) ou escolha outro clipe.",
"ref_audio_too_long": "Uma referência com transcrição fornecida pode ter no máximo 20 segundos. Corte o áudio e a transcrição para o mesmo trecho de 3–10 segundos ou remova a transcrição para o VoiceStudio cortar e transcrever o clipe automaticamente."
"ref_audio_too_long": "Uma referência pode ter no máximo 20 segundos com transcrição ou 75 segundos sem ela. Corte-a para um trecho de fala claro de 3–10 segundos; se fornecer texto, corte a transcrição para o mesmo trecho.",
"ref_audio_no_speech": "A detecção automática não encontrou palavras faladas na referência. Corte-a para um trecho de fala claro de 3–10 segundos ou forneça uma transcrição correspondente."
},
"sharing": {
"title": "Compartilhamento e acesso remoto",
frontend/src/i18n/locales/ru.json
+2 -1
View File
@@ -2059,7 +2059,8 @@
"ignored_unsupported": "Игнорируется неподдерживаемая инструкция: {{items}}",
"ignored_duplicate": "Игнорируется (категория уже установлена): {{items}}",
"ref_audio_unusable": "В референсном аудио не слышно речи — клип тихий или полностью беззвучный, клонировать нечего. Запишите заново ближе к микрофону (проверьте, что выбрано правильное устройство ввода и уровень записи движется) или выберите другой клип.",
"ref_audio_too_long": "Референс с готовой расшифровкой должен быть не длиннее 20 секунд. Обрежьте аудио и текст до одного и того же фрагмента длиной 3–10 секунд или удалите текст, чтобы VoiceStudio автоматически обрезал и расшифровал клип."
"ref_audio_too_long": "Референс может длиться не более 20 секунд с расшифровкой или 75 секунд без неё. Обрежьте его до чёткого речевого фрагмента длиной 3–10 секунд; если добавляете текст, обрежьте расшифровку до того же фрагмента.",
"ref_audio_no_speech": "Автоматическое распознавание не обнаружило речи в референсе. Обрежьте его до чёткого речевого фрагмента длиной 3–10 секунд или добавьте соответствующую расшифровку."
},
"sharing": {
"title": "Совместное использование и удаленный доступ",
frontend/src/i18n/locales/sv.json
+2 -1
View File
@@ -2059,7 +2059,8 @@
"ignored_unsupported": "Ignorerad instruktion som inte stöds: {{items}}",
"ignored_duplicate": "Ignorerad (kategori redan inställd): {{items}}",
"ref_audio_unusable": "Inget tal kunde höras i referensljudet — klippet är tyst eller nästan tyst, så det finns ingen röst att klona. Spela in igen närmare mikrofonen (kontrollera att rätt ingångsenhet är vald och att nivån rör sig) eller välj ett annat klipp.",
"ref_audio_too_long": "En referens med angiven transkription får vara högst 20 sekunder. Klipp ljud och transkription till samma avsnitt på 3–10 sekunder, eller ta bort transkriptionen så att VoiceStudio kan klippa och transkribera automatiskt."
"ref_audio_too_long": "En referens får vara högst 20 sekunder med transkription eller 75 sekunder utan. Klipp den till ett tydligt talavsnitt på 3–10 sekunder; om du anger text ska transkriptionen klippas till samma avsnitt.",
"ref_audio_no_speech": "Den automatiska taldetekteringen hittade inga talade ord. Klipp referensen till ett tydligt talavsnitt på 3–10 sekunder eller ange en matchande transkription."
},
"sharing": {
"title": "Delning och fjärråtkomst",
frontend/src/i18n/locales/th.json
+2 -1
View File
@@ -2059,7 +2059,8 @@
"ignored_unsupported": "ละเว้นคำสั่งที่ไม่สนับสนุน: {{items}}",
"ignored_duplicate": "ละเว้น (หมวดหมู่ที่ตั้งไว้แล้ว): {{items}}",
"ref_audio_unusable": "ไม่ได้ยินเสียงพูดในไฟล์เสียงอ้างอิง — คลิปเงียบหรือเกือบไม่มีเสียง จึงไม่มีเสียงให้โคลน โปรดอัดใหม่ให้ใกล้ไมโครโฟนมากขึ้น (ตรวจสอบว่าเลือกอุปกรณ์รับเสียงถูกต้อง) หรือเลือกคลิปอื่น",
"ref_audio_too_long": "เสียงอ้างอิงที่มีข้อความถอดเสียงต้องยาวไม่เกิน 20 วินาที ตัดทั้งเสียงและข้อความให้เป็นช่วงเดียวกัน 3–10 วินาที หรือลบข้อความเพื่อให้ VoiceStudio ตัดและถอดเสียงให้อัตโนมัติ"
"ref_audio_too_long": "เสียงอ้างอิงยาวได้ไม่เกิน 20 วินาทีเมื่อมีข้อความถอดเสียง หรือ 75 วินาทีเมื่อไม่มี ตัดให้เหลือช่วงคำพูดชัดเจน 3–10 วินาที และหากมีข้อความให้ตัดตรงกับช่วงเดียวกัน",
"ref_audio_no_speech": "การตรวจจับอัตโนมัติไม่พบคำพูดในเสียงอ้างอิง ตัดให้เหลือช่วงคำพูดชัดเจน 3–10 วินาที หรือใส่ข้อความถอดเสียงที่ตรงกัน"
},
"sharing": {
"title": "การแชร์และการเข้าถึงระยะไกล",
frontend/src/i18n/locales/tr.json
+2 -1
View File
@@ -2059,7 +2059,8 @@
"ignored_unsupported": "Desteklenmeyen talimat göz ardı edildi: {{items}}",
"ignored_duplicate": "Yoksayıldı (kategori zaten ayarlandı): {{items}}",
"ref_audio_unusable": "Referans seste konuşma duyulamadı — klip sessiz ya da neredeyse sessiz, bu yüzden klonlanacak bir ses yok. Mikrofona daha yakın şekilde yeniden kayıt yapın (doğru giriş aygıtının seçili olduğundan ve seviyenin hareket ettiğinden emin olun) veya başka bir klip seçin.",
"ref_audio_too_long": "Transkripti verilen bir referans en fazla 20 saniye olabilir. Ses ve transkripti aynı 3–10 saniyelik bölüme kırpın veya VoiceStudio'nun otomatik kırpıp yazıya dökmesi için transkripti kaldırın."
"ref_audio_too_long": "Bir referans, transkriptle en fazla 20 saniye; transkriptsiz en fazla 75 saniye olabilir. Sesi 3–10 saniyelik net bir konuşma bölümüne kırpın; metin sağlıyorsanız transkripti de aynı bölüme kırpın.",
"ref_audio_no_speech": "Otomatik konuşma algılama referansta söylenmiş kelime bulamadı. Sesi 3–10 saniyelik net bir konuşma bölümüne kırpın veya eşleşen bir transkript sağlayın."
},
"sharing": {
"title": "Paylaşım ve Uzaktan Erişim",
frontend/src/i18n/locales/uk.json
+2 -1
View File
@@ -2059,7 +2059,8 @@
"ignored_unsupported": "Ігнорована непідтримувана інструкція: {{items}}",
"ignored_duplicate": "Проігноровано (категорію вже встановлено): {{items}}",
"ref_audio_unusable": "У референсному аудіо не чути мовлення — кліп тихий або повністю беззвучний, тож голосу для клонування немає. Запишіть ще раз ближче до мікрофона (перевірте, чи вибрано правильний пристрій введення і чи рухається рівень) або виберіть інший кліп.",
"ref_audio_too_long": "Референс із готовою транскрипцією має бути не довшим за 20 секунд. Обріжте аудіо й текст до того самого уривка тривалістю 3–10 секунд або видаліть текст, щоб VoiceStudio автоматично обрізав і транскрибував кліп."
"ref_audio_too_long": "Референс може тривати не більше 20 секунд із транскрипцією або 75 секунд без неї. Обріжте його до чіткого мовного уривка тривалістю 3–10 секунд; якщо додаєте текст, обріжте транскрипцію до того самого уривка.",
"ref_audio_no_speech": "Автоматичне розпізнавання не виявило мовлення в референсі. Обріжте його до чіткого мовного уривка тривалістю 3–10 секунд або додайте відповідну транскрипцію."
},
"sharing": {
"title": "Спільне використання та віддалений доступ",
frontend/src/i18n/locales/vi.json
+2 -1
View File
@@ -2059,7 +2059,8 @@
"ignored_unsupported": "Đã bỏ qua hướng dẫn không được hỗ trợ: {{items}}",
"ignored_duplicate": "Đã bỏ qua (danh mục đã được đặt): {{items}}",
"ref_audio_unusable": "Không nghe thấy giọng nói trong âm thanh tham chiếu — đoạn clip im lặng hoặc gần như im lặng nên không có giọng để nhân bản. Hãy thu âm lại gần micro hơn (kiểm tra đã chọn đúng thiết bị đầu vào và mức âm có dao động) hoặc chọn một clip khác.",
"ref_audio_too_long": "Âm thanh tham chiếu có bản chép lời chỉ được dài tối đa 20 giây. Hãy cắt cả âm thanh và văn bản về cùng đoạn 3–10 giây, hoặc xóa văn bản để VoiceStudio tự động cắt và chép lời clip."
"ref_audio_too_long": "Âm thanh tham chiếu dài tối đa 20 giây khi có bản chép lời hoặc 75 giây khi không có. Hãy cắt thành đoạn lời nói rõ dài 3–10 giây; nếu cung cấp văn bản, hãy cắt bản chép lời theo đúng đoạn đó.",
"ref_audio_no_speech": "Tính năng phát hiện tự động không tìm thấy lời nói trong âm thanh tham chiếu. Hãy cắt thành đoạn lời nói rõ dài 3–10 giây hoặc cung cấp bản chép lời khớp."
},
"sharing": {
"title": "Chia sẻ & Truy cập từ xa",
frontend/src/i18n/locales/zh-CN.json
+2 -1
View File
@@ -2066,7 +2066,8 @@
"ignored_unsupported": "忽略不支持的指令：{{items}}",
"ignored_duplicate": "忽略（类别已设置）：{{items}}",
"ref_audio_unusable": "参考音频中听不到任何语音——片段是静音或几乎静音的，因此没有可克隆的声音。请靠近麦克风重新录制（检查是否选择了正确的输入设备、电平是否有跳动），或换一段其他音频。",
"ref_audio_too_long": "带有文字稿的参考音频最长为20秒。请将音频和文字稿裁剪为同一个3–10秒片段，或删除文字稿，让VoiceStudio自动裁剪并转写音频。"
"ref_audio_too_long": "参考音频在提供文字稿时最长20秒，未提供时最长75秒。请裁剪为含清晰语音的3–10秒片段；若提供文字稿，请将其裁剪到同一片段。",
"ref_audio_no_speech": "自动语音检测未在参考音频中发现口语。请裁剪为含清晰语音的3–10秒片段，或提供匹配的文字稿。"
},
"sharing": {
"title": "共享和远程访问",
frontend/src/i18n/locales/zh-TW.json
+2 -1
View File
@@ -2059,7 +2059,8 @@
"ignored_unsupported": "忽略不支援的指令：{{items}}",
"ignored_duplicate": "忽略（類別已設定）：{{items}}",
"ref_audio_unusable": "參考音訊中聽不到任何語音——片段為靜音或幾乎靜音，因此沒有可複製的聲音。請靠近麥克風重新錄製（檢查是否選擇了正確的輸入裝置、音量電平是否有跳動），或改用其他片段。",
"ref_audio_too_long": "附有文字稿的參考音訊最長為20秒。請將音訊和文字稿裁剪為同一個3–10秒片段，或移除文字稿，讓VoiceStudio自動裁剪並轉寫音訊。"
"ref_audio_too_long": "參考音訊在提供文字稿時最長20秒，未提供時最長75秒。請裁剪為含清晰語音的3–10秒片段；若提供文字稿，請將其裁剪到同一片段。",
"ref_audio_no_speech": "自動語音偵測未在參考音訊中找到口語。請裁剪為含清晰語音的3–10秒片段，或提供相符的文字稿。"
},
"sharing": {
"title": "共享和遠端存取",
frontend/src/test/errorToastUserFixable.test.jsx
+9
View File
@@ -55,6 +55,14 @@ describe('toastErrorWithReport user-fixable marker mapping (#1188)', () => {
});
});
it('shows localized guidance when bounded selection finds no speech', () => {
const detail = '[clone_ref_no_speech] Automatic speech detection found no spoken words.';
toastErrorWithReport(detail, new Error(detail));
expect(toastErrorMock).toHaveBeenCalledWith('t:tts_errors.ref_audio_no_speech', {
duration: 8000,
});
});
it('unmarked errors keep the Report-action toast (JSX renderer, not a plain string)', () => {
toastErrorWithReport('Error: something exploded', new Error('something exploded'));
expect(toastErrorMock).toHaveBeenCalledTimes(1);
@@ -69,6 +77,7 @@ describe('toastErrorWithReport user-fixable marker mapping (#1188)', () => {
const locale = JSON.parse(fs.readFileSync(path.join(localesDir, f), 'utf8'));
expect(locale.tts_errors?.ref_audio_unusable, `${f} missing the key`).toBeTruthy();
expect(locale.tts_errors?.ref_audio_too_long, `${f} missing the key`).toBeTruthy();
expect(locale.tts_errors?.ref_audio_no_speech, `${f} missing the key`).toBeTruthy();
}
});
});
frontend/src/test/streamingTts.test.js
+16
View File
@@ -19,6 +19,7 @@ const {
decodePcm16Base64,
peaksFromChunkList,
StreamingPreviewError,
shouldFallbackToClassic,
} = await import('../utils/streamingTts');
const { getPlaybackTrack, stopActivePlayback, seekActivePlayback } =
await import('../utils/playback');
@@ -312,6 +313,21 @@ describe('streamGenerateSpeech', () => {
expect(plain.retryable).toBe(false);
});
it('marks actionable clone-reference errors terminal to prevent a classic retry', async () => {
apiFetch.mockResolvedValue(
ndjsonResponse([
startEvent(1),
{ type: 'error', detail: '[clone_ref_no_speech] trim the reference' },
]),
);
const err = await streamGenerateSpeech(new FormData(), {}).catch((e) => e);
expect(err).toBeInstanceOf(StreamingPreviewError);
expect(err.terminal).toBe(true);
expect(shouldFallbackToClassic(err)).toBe(false);
});
it('rejects with StreamingPreviewError when the stream ends without done', async () => {
apiFetch.mockResolvedValue(ndjsonResponse([startEvent(2), chunkEvent(0)]));
await expect(streamGenerateSpeech(new FormData(), {})).rejects.toThrow(StreamingPreviewError);
frontend/src/utils/errorToast.jsx
+1
View File
@@ -28,6 +28,7 @@ import { openBugReport } from './bugReport';
const USER_FIXABLE_MARKERS = {
'[clone_ref_unusable]': 'tts_errors.ref_audio_unusable',
'[clone_ref_too_long]': 'tts_errors.ref_audio_too_long',
'[clone_ref_no_speech]': 'tts_errors.ref_audio_no_speech',
'[shutting_down]': 'errors.backend_shutting_down',
};
frontend/src/utils/streamingTts.js
+12 -1
View File
@@ -79,9 +79,13 @@ export class StreamingPreviewError extends Error {
// the same error. Carried here so useTTS can tell the two apart.
this.retryable = opts?.retryable === true;
this.retryAfter = opts?.retryAfter ?? null;
this.terminal = opts?.terminal === true;
}
}
export const shouldFallbackToClassic = (error) =>
error instanceof StreamingPreviewError && !error.retryable && !error.terminal;
/** base64 → Int16 PCM → Float32 samples in [-1, 1]. Exported for tests. */
export const decodePcm16Base64 = (b64) => {
const bin = atob(b64);
@@ -380,9 +384,16 @@ async function _streamGenerateSpeech(
} else if (ev.type === 'done') {
meta = ev;
} else if (ev.type === 'error') {
throw new StreamingPreviewError(ev.detail || 'TTS stream reported an error', {
const detail = ev.detail || 'TTS stream reported an error';
const terminal = [
'[clone_ref_unusable]',
'[clone_ref_too_long]',
'[clone_ref_no_speech]',
].some((marker) => detail.includes(marker));
throw new StreamingPreviewError(detail, {
retryable: ev.retryable === true,
retryAfter: ev.retry_after ?? null,
terminal,
});
}
};
omnivoice/models/omnivoice.py
+24 -60
View File
@@ -53,12 +53,12 @@ from transformers.modeling_outputs import ModelOutput
from transformers.models.auto import CONFIG_MAPPING, AutoConfig
from omnivoice.utils.audio import (
CLONE_REF_NO_SPEECH_MARKER,
CLONE_REF_TOO_LONG_MARKER,
cross_fade_chunks,
fade_and_pad_audio,
load_audio,
remove_silence_safe,
trim_long_audio,
CLONE_REF_TOO_LONG_MARKER,
validate_clone_reference,
)
from omnivoice.utils.duration import RuleDurationEstimator
@@ -798,85 +798,49 @@ class OmniVoice(PreTrainedModel):
"same 3-10 second passage, or omit the transcript so VoiceStudio "
"can trim and transcribe the clip automatically."
)
# Safety bound, independent of the caller's optional silence/
# punctuation preprocessing choice. The transcript is generated
# from this trimmed waveform below, so audio and text stay aligned.
untrimmed_ref_wav = ref_wav
ref_wav = trim_long_audio(ref_wav, self.sampling_rate, trim_threshold=20.0)
# ``trim_long_audio`` splits at speech/silence boundaries and is
# deliberately fail-open when it cannot detect speech. Safety is
# not optional here: fall back to the same 15 s upper bound so an
# unusual/quiet clip still cannot allocate tokens for minutes.
# Transcript-free automatic selection examines every passage, but
# caps the work at five bounded ASR calls. Longer references need
# an explicit user-selected passage rather than a lossy sampling
# policy that could silently miss speech between fixed windows.
max_samples = int(15.0 * self.sampling_rate)
original_power = untrimmed_ref_wav.abs().amax(dim=0).square()
max_auto_samples = 5 * max_samples
if ref_wav.size(-1) > max_auto_samples:
raise ValueError(
f"{CLONE_REF_TOO_LONG_MARKER} Reference audio is "
f"{ref_duration:.1f} seconds long; automatic transcript-free "
"selection supports at most 75 seconds. Trim the audio to a "
"3-10 second speech passage, or supply a matching transcript."
)
original_power = ref_wav.abs().amax(dim=0).square()
activity_cap = max(float(original_power.mean()) * 100.0, 1e-10)
def activity_score(audio):
power = audio.abs().amax(dim=0).square().clamp_max(activity_cap)
return float(power.double().sum())
def densest_window(audio):
envelope = audio.abs().amax(dim=0)
# Pick the contiguous passage with the most activity. Using
# the midpoint of the first/last crossings can center the
# crop on silence when a transient is far from real speech.
power = envelope.square().clamp_max(activity_cap)
# Float64 keeps small quiet-speech contributions observable
# after a distant full-scale transient in long references.
cumulative = torch.nn.functional.pad(power.double().cumsum(0), (1, 0))
window_power = cumulative[max_samples:] - cumulative[:-max_samples]
start = int(window_power.argmax())
return audio[:, start : start + max_samples], float(window_power[start])
best_original, _ = densest_window(untrimmed_ref_wav)
if ref_wav.size(-1) > max_samples:
ref_wav, _ = densest_window(ref_wav)
# Energy is only a fallback: music/noise can be louder than the
# actual speaker. Ask the already-required ASR boundary which
# bounded passage contains speech, then keep that exact waveform
# and transcript aligned for tokenization below.
if self._asr_pipe is None:
logger.info("ASR model not loaded yet, loading on-the-fly ...")
self.load_asr_model()
max_asr_candidates = 5
candidates = [ref_wav, best_original]
last_start = untrimmed_ref_wav.size(-1) - max_samples
# Bound Whisper work for arbitrarily long files while retaining
# whole-file coverage, including the far end where late speech is
# commonly found. The trim result and energy-best passage occupy
# the first two slots; evenly spaced windows fill the rest.
sampled_starts = torch.linspace(
0,
last_start,
steps=max_asr_candidates,
dtype=torch.int64,
).tolist()
starts = [sampled_starts[-1], sampled_starts[0], *sampled_starts[1:-1]]
candidates.extend(
untrimmed_ref_wav[:, start : start + max_samples] for start in starts
)
unique_candidates = []
seen = set()
for candidate in candidates:
identity = (candidate.data_ptr(), candidate.storage_offset(), candidate.size(-1))
if identity not in seen:
seen.add(identity)
unique_candidates.append(candidate)
if len(unique_candidates) == max_asr_candidates:
break
candidates = list(ref_wav.split(max_samples, dim=-1))
def speech_score(text):
return len(re.sub(r"[^\w]+", "", text or "", flags=re.UNICODE))
transcribed = [
(self.transcribe((candidate, self.sampling_rate)), candidate)
for candidate in unique_candidates
for candidate in candidates
]
ref_text, ref_wav = max(
transcribed,
key=lambda item: (speech_score(item[0]), activity_score(item[1])),
)
if speech_score(ref_text) == 0:
raise ValueError(
f"{CLONE_REF_NO_SPEECH_MARKER} Automatic speech detection "
"could not find spoken words in the reference. Trim it to a "
"clear 3-10 second speech passage, or supply a matching transcript."
)
if preprocess_prompt:
# #1188: the fixed -50 dBFS silence threshold used to consume a
omnivoice/utils/audio.py
+1
View File
@@ -39,6 +39,7 @@ logger = logging.getLogger(__name__)
# exact string (frontend/src/utils/errorToast.jsx) — change both or neither.
CLONE_REF_UNUSABLE_MARKER = "[clone_ref_unusable]"
CLONE_REF_TOO_LONG_MARKER = "[clone_ref_too_long]"
CLONE_REF_NO_SPEECH_MARKER = "[clone_ref_no_speech]"
def load_audio(audio_path: str, sampling_rate: int):
tests/test_omnivoice_reference_bound.py
+58 -35
View File
@@ -54,14 +54,10 @@ def test_supplied_transcript_rejects_long_file_before_tokenization(tmp_path):
)
def test_missing_transcript_is_safely_trimmed_even_when_preprocessing_is_disabled(monkeypatch):
def test_missing_transcript_is_safely_trimmed_even_when_preprocessing_is_disabled():
model = _model(reject_tokenization=False)
model._asr_pipe = object()
model.transcribe = lambda _audio: "Automatically aligned transcript."
monkeypatch.setattr(
"omnivoice.models.omnivoice.trim_long_audio",
lambda audio, sampling_rate, **_kwargs: audio[:, : 15 * sampling_rate],
)
audio = torch.full((1, 21 * 24_000), 0.1)
model.create_voice_clone_prompt(
@@ -70,32 +66,24 @@ def test_missing_transcript_is_safely_trimmed_even_when_preprocessing_is_disable
preprocess_prompt=False,
)
assert model.audio_tokenizer.seen_samples == 15 * 24_000
assert model.audio_tokenizer.seen_samples <= 15 * 24_000
def test_missing_transcript_still_has_a_hard_bound_when_silence_split_cannot_trim(monkeypatch):
def test_missing_transcript_still_has_a_hard_bound_when_silence_split_cannot_trim():
model = _model(reject_tokenization=False)
model._asr_pipe = object()
model.transcribe = lambda _audio: "Automatically aligned transcript."
monkeypatch.setattr(
"omnivoice.models.omnivoice.trim_long_audio",
lambda audio, _sampling_rate, **_kwargs: audio,
)
audio = torch.full((1, 21 * 24_000), 0.1)
model.create_voice_clone_prompt((audio, 24_000), ref_text=None)
assert model.audio_tokenizer.seen_samples == 15 * 24_000
assert model.audio_tokenizer.seen_samples <= 15 * 24_000
def test_hard_bound_keeps_late_speech_instead_of_cropping_only_silence(monkeypatch):
model = _model(reject_tokenization=False)
model._asr_pipe = object()
model.transcribe = lambda _audio: "Automatically aligned transcript."
monkeypatch.setattr(
"omnivoice.models.omnivoice.trim_long_audio",
lambda audio, _sampling_rate, **_kwargs: audio,
)
monkeypatch.setattr(
"omnivoice.models.omnivoice.remove_silence_safe",
lambda audio, *_args, **_kwargs: audio,
@@ -105,7 +93,7 @@ def test_hard_bound_keeps_late_speech_instead_of_cropping_only_silence(monkeypat
model.create_voice_clone_prompt((audio, 24_000), ref_text=None)
assert model.audio_tokenizer.seen_samples == 15 * 24_000
assert model.audio_tokenizer.seen_samples <= 15 * 24_000
assert model.audio_tokenizer.seen_peak > 0
@@ -113,10 +101,6 @@ def test_hard_bound_prefers_speech_over_distant_transient(monkeypatch):
model = _model(reject_tokenization=False)
model._asr_pipe = object()
model.transcribe = lambda _audio: "Automatically aligned transcript."
monkeypatch.setattr(
"omnivoice.models.omnivoice.trim_long_audio",
lambda audio, _sampling_rate, **_kwargs: audio,
)
monkeypatch.setattr(
"omnivoice.models.omnivoice.remove_silence_safe",
lambda audio, *_args, **_kwargs: audio,
@@ -127,7 +111,7 @@ def test_hard_bound_prefers_speech_over_distant_transient(monkeypatch):
model.create_voice_clone_prompt((audio, 24_000), ref_text=None)
assert model.audio_tokenizer.seen_samples == 15 * 24_000
assert model.audio_tokenizer.seen_samples <= 15 * 24_000
# Quiet references are RMS-normalized before cropping. The speech peak is
# therefore non-zero but remains far below the isolated transient.
assert 0 < model.audio_tokenizer.seen_peak < 1
@@ -146,7 +130,7 @@ def test_real_silence_splitter_does_not_keep_silent_prefix(monkeypatch):
model.create_voice_clone_prompt((audio, 24_000), ref_text=None)
assert model.audio_tokenizer.seen_samples == 15 * 24_000
assert model.audio_tokenizer.seen_samples <= 15 * 24_000
assert model.audio_tokenizer.seen_peak > 0
@@ -164,7 +148,7 @@ def test_real_silence_splitter_ignores_transient_before_late_speech(monkeypatch)
model.create_voice_clone_prompt((audio, 24_000), ref_text=None)
assert model.audio_tokenizer.seen_samples == 15 * 24_000
assert model.audio_tokenizer.seen_samples <= 15 * 24_000
assert 0 < model.audio_tokenizer.seen_peak < 1
@@ -174,10 +158,6 @@ def test_speech_aware_bound_prefers_quiet_voice_over_loud_non_speech(monkeypatch
model.transcribe = lambda candidate: (
"Spoken words." if float(candidate[0].abs().mean()) < 0.05 else ""
)
monkeypatch.setattr(
"omnivoice.models.omnivoice.trim_long_audio",
lambda audio, _sampling_rate, **_kwargs: audio,
)
monkeypatch.setattr(
"omnivoice.models.omnivoice.remove_silence_safe",
lambda audio, *_args, **_kwargs: audio,
@@ -191,7 +171,7 @@ def test_speech_aware_bound_prefers_quiet_voice_over_loud_non_speech(monkeypatch
assert model.audio_tokenizer.seen_peak < 0.1
def test_far_late_speech_survives_with_bounded_asr_calls(monkeypatch):
def test_far_late_speech_at_auto_select_limit_survives_with_five_asr_calls(monkeypatch):
model = _model(reject_tokenization=False)
model.sampling_rate = 100
model._asr_pipe = object()
@@ -202,18 +182,61 @@ def test_far_late_speech_survives_with_bounded_asr_calls(monkeypatch):
return "Late speech." if float(candidate[0][0, -1]) > 0 else ""
model.transcribe = transcribe
monkeypatch.setattr(
"omnivoice.models.omnivoice.trim_long_audio",
lambda audio, sampling_rate, **_kwargs: audio[:, : 15 * sampling_rate],
)
monkeypatch.setattr(
"omnivoice.models.omnivoice.remove_silence_safe",
lambda audio, *_args, **_kwargs: audio,
)
audio = torch.zeros((1, 184 * 100))
audio = torch.zeros((1, 75 * 100))
audio[:, -5 * 100 :] = 0.02
prompt = model.create_voice_clone_prompt((audio, 100), ref_text=None)
assert prompt.ref_text == "Late speech."
assert len(calls) <= 5
assert len(calls) == 5
def test_auto_select_rejects_one_sample_over_limit_before_asr():
model = _model(reject_tokenization=False)
model.sampling_rate = 100
model._asr_pipe = object()
calls = []
model.transcribe = lambda candidate: calls.append(candidate) or "Speech."
audio = torch.full((1, 75 * 100 + 1), 0.1)
with pytest.raises(ValueError, match=r"\[clone_ref_too_long\].*at most 75 seconds"):
model.create_voice_clone_prompt((audio, 100), ref_text=None)
assert calls == []
def test_auto_select_tiles_final_partial_window_without_gaps(monkeypatch):
model = _model(reject_tokenization=False)
model.sampling_rate = 100
model._asr_pipe = object()
lengths = []
def transcribe(candidate):
lengths.append(candidate[0].size(-1))
return "Speech."
model.transcribe = transcribe
monkeypatch.setattr(
"omnivoice.models.omnivoice.remove_silence_safe",
lambda audio, *_args, **_kwargs: audio,
)
audio = torch.full((1, 61 * 100), 0.1)
model.create_voice_clone_prompt((audio, 100), ref_text=None)
assert lengths == [1500, 1500, 1500, 1500, 100]
def test_auto_select_rejects_punctuation_only_transcripts_before_tokenizer():
model = _model(reject_tokenization=True)
model.sampling_rate = 100
model._asr_pipe = object()
model.transcribe = lambda _candidate: "...?!"
audio = torch.full((1, 21 * 100), 0.1)
with pytest.raises(ValueError, match=r"\[clone_ref_no_speech\].*could not find spoken words"):
model.create_voice_clone_prompt((audio, 100), ref_text=None)