Palash Debnath
276875c397
The longform marker dialect (# heading / [voice:] / [pause] / SSML-lite) was parsed by three independent code paths that already disagreed (client vs server on [pause] units, [voice:] empty, H1-only chapters). This lands the single canonical Python parser; the JS port + cross-impl test follow in slice B. - New backend/services/longform_parser.py — parse_script_to_spans(text, *, default_voice, default_speed) + _parse_chapter_body (the reusable voice→pause →SSML layering the JS twin mirrors). Moves the H1/voice regexes verbatim from audiobook.py (already CodeQL-cleared), reuses parse_pause_markers + ssml_lite unchanged. Coerces None→"" and normalizes CRLF/CR→LF at entry (cross-platform parity so Windows-authored scripts never carry a stray \r). Adds default_speed plumbing (inline SSML speed overrides the per-line default). - audiobook.py: parse_audiobook_script is now a thin wrapper that wraps the canonical span dicts in Span/Chapter/AudiobookPlan — public return type and .to_dict() shape unchanged, all four router call sites untouched. Deleted _parse_spans / _HEADING_RE / _VOICE_RE and the now-dead `import re` + parse_pause_markers import. - tests/fixtures/longform_parser_cases.json — 78-case golden corpus (≥40 required) covering §A–I: H1-only chapters (H2–H6 + `# ` no-title → body), the full pause dialect incl. the NO-MATCH boundary, banker's-rounding ties ([pause 0.5]→0, [pause 1.5]→2), [voice:] empty→default, [voice:[nested]] literal, SSML nesting/spell/unknown-tag, speed override, CRLF, combined precedence. Generated from actual parser output (the truth the JS port must match). - tests/test_longform_parser.py — parametrized over the corpus + None-input + ReDoS-linearity (5000× repeats < 1 s). 130 passed (corpus + test_audiobook + test_pause_markers + test_ssml_lite all green); CJK guard green.
49 lines
1.6 KiB
Python
49 lines
1.6 KiB
Python
"""Canonical longform parser (#27) — pytest side of the cross-impl golden corpus.
|
||
|
||
This suite and ``frontend/src/test/longformParser.test.js`` load the SAME JSON
|
||
(`tests/fixtures/longform_parser_cases.json`) and assert both impls produce it
|
||
byte-for-byte. A divergence cannot pass both suites — the side that drifts fails
|
||
its own assertion against the shared truth.
|
||
"""
|
||
from __future__ import annotations
|
||
|
||
import json
|
||
import pathlib
|
||
|
||
import pytest
|
||
|
||
from services.longform_parser import parse_script_to_spans
|
||
|
||
_FIXTURE = pathlib.Path(__file__).parent / "fixtures" / "longform_parser_cases.json"
|
||
_CASES = json.loads(_FIXTURE.read_text(encoding="utf-8"))
|
||
|
||
|
||
@pytest.mark.parametrize("case", _CASES, ids=[c["name"] for c in _CASES])
|
||
def test_corpus(case):
|
||
got = parse_script_to_spans(
|
||
case["input"],
|
||
default_voice=case["default_voice"],
|
||
default_speed=case.get("default_speed"),
|
||
)
|
||
assert got == case["expected"]
|
||
|
||
|
||
def test_none_input_returns_empty():
|
||
# The wrapper coerces, but the parser itself must not raise on None.
|
||
assert parse_script_to_spans(None) == []
|
||
|
||
|
||
def test_corpus_has_enough_cases():
|
||
# The spec mandates ≥40 cases covering §A–I.
|
||
assert len(_CASES) >= 40
|
||
|
||
|
||
def test_pathological_inputs_are_linear():
|
||
# ReDoS guard: adversarial repeats must finish fast (mirrors the JS suite).
|
||
import time
|
||
for blob in ("[slow]" * 5000, "[pause" * 5000, "[voice:" * 5000,
|
||
"# \n" * 5000, "[a]" * 5000):
|
||
t0 = time.perf_counter()
|
||
parse_script_to_spans(blob, default_voice="v")
|
||
assert time.perf_counter() - t0 < 1.0
|