Review follow-up: packaged desktop installs do not ship alembic.ini, so the
locale decode only reaches from-source runs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Dub -> Paste translation -> Load file accepts .vtt and sends timestamped
text to the lenient SRT parser, which is meant to take VTT too. Two
ordinary WebVTT files broke it:
- Cues without an hours field (00:01.000 --> 00:04.500) matched neither
the frontend's timing detector nor the backend pattern, so the dialog
mapped WEBVTT, the timing lines and the dialogue as plain translations,
and the endpoint itself answered "No timed cues found".
- A cue identifier or NOTE block after a cue became part of that cue's
text, because only digit-only index lines were trimmed.
The hours are now optional in both patterns, as dub_pipeline's yt-dlp
caption parser already allows. For WebVTT input a cue's text ends at its
first blank line, as the format specifies; SRT keeps its lenient
blank-line handling.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
alembic reads alembic.ini with encoding="locale". On Python 3.11 that is
the Windows ANSI code page even under PYTHONUTF8=1, which the desktop
shell sets, so the em dashes in the ini's comments raised
UnicodeDecodeError inside Config() on cp932/cp936/cp949/cp950 systems.
_run_alembic_upgrade treats that as "alembic unavailable": every startup
logged "alembic upgrade head skipped: 'cp949' codec can't decode byte
0xe2", never took the pre-migration backup, and never ran a migration --
including the data-healing ones (0006/0007) that the additive column
reconcile cannot replace.
The ini is now ASCII, with a note saying why, and a test parses it in
each of those code pages with the parser alembic builds and pins it
ASCII. Same Python 3.11 locale-decoding class as the .pth fix in #1795.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The SRT/VTT formatters in dub_export and openai_compat truncated
(seconds % 1) * 1000. Most decimal times are not exact in binary (2.3 is
2.29999...), so a cue imported as 00:00:02,300 exported as 00:00:02,299:
every such cue moved a millisecond early in the /dub/srt and /dub/vtt
downloads, burned-in subtitles, and /v1/audio/transcriptions srt/vtt.
All four now call srt_parser.format_cue_timestamp, which rounds the whole
value to milliseconds once and splits it, so 59.9996 carries to
00:01:00,000 rather than printing ",1000" -- the same round-then-divmod
shape karaoke_ass._ass_time already uses.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Review follow-up. A Windows-1252 upload with one of the five bytes cp1252
leaves undefined fell back to decoding the entire file as Latin-1, so its
curly quotes, dashes and euro signs became C1 control characters. Only
the undefined bytes now take their Latin-1 code point, which is what the
browser's windows-1252 decoder does, so readTextFile and
decode_text_upload agree byte for byte.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Stories -> Import read the picked file with File.text(), which decodes
UTF-8 only, so a UTF-16 script came back NUL-riddled and a Windows-1252
one as replacement characters. Dub -> Paste translation -> Load file used
FileReader.readAsText(), which handles a UTF-16 BOM but still turned
Windows-1252 accents, dashes and quotes into replacement characters.
Both now go through readTextFile, which applies decode_text_upload's
rule with TextDecoder: a BOM names the encoding, valid UTF-8 stays
UTF-8, anything else is Windows-1252.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
/dub/import-srt fell back to Latin-1 when UTF-8 failed, so a UTF-16 .srt
(Notepad's "Unicode", many subtitle editors) decoded with a NUL between
every character and was rejected as having no cues, and a Windows-1252
one turned curly quotes and dashes into C1 control characters.
/audiobook/import decoded .txt/.md with errors="ignore", silently dropping
every accent, dash and curly quote from a Windows-1252 manuscript,
returning NUL-interleaved text for a UTF-16 one, and keeping a UTF-8 BOM
at the start of the editor text.
Both now use decode_text_upload: a BOM names the encoding, valid UTF-8
stays UTF-8, and anything else is read as Windows-1252, with Latin-1 for
the bytes cp1252 leaves undefined so the decode never raises.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
