Project-Thoth/applications/chatgpt-capture/codex-work-orders/debugging/0002-block-defects.md

Fix ChatGPT DOM extractor block-structure defects discovered during manual testing.

Observed defect: Large assistant responses are being flattened into plain lines. Paragraph breaks, list structure, indentation, and inline emphasis boundaries are being lost.

Example bad output: There's a real pattern here, but it's useful to separate public narratives from the underlying issues . Over the past several decades...

Expected behavior: There's a real pattern here, but it's useful to separate public narratives from the underlying issues.

Over the past several decades, there have indeed been many highly publicized predictions of catastrophic outcomes:

• Nuclear war during the Cold War.
• Concerns about global cooling in the 1970s...
• Y2K causing widespread infrastructure failures.

Root cause: The extractor is likely using innerText/textContent or recursively joining nodes without distinguishing inline elements from block elements.

Requirements:

1. Update src/chatgptExtractor.js.

2. Implement DOM-to-Markdown extraction that treats block and inline elements differently:

   • Inline elements must remain inline.
   • Block elements must create paragraph/list/code boundaries.
   • Do not insert line breaks around inline tags like strong, em, span, a, code.

3. Preserve paragraphs:

   • Each

     becomes one paragraph.

   • Paragraphs separated by one blank line.
   • Inline formatting inside paragraphs must remain inline.

4. Preserve unordered lists:

   • • Item
     becomes: - Item
   • Preserve nested list indentation where practical.

5. Preserve ordered lists:

   • 1. Item
     becomes: 1. Item 2. Item

6. Preserve blockquotes where detectable:

   • Prefix quoted block lines with >.

7. Preserve headings:

   • h1-h6 become Markdown headings.

8. Preserve code blocks:

   • ...

     becomes fenced code.
   • Do not wrap or reformat code content.

9. Preserve inline formatting:

   • / → text
   • / → text
   • inline → text
   • may preserve readable text only for MVP.

10. Add a cleaner that operates after Markdown generation:

• Remove punctuation-only lines caused by extraction.
• Collapse 3+ blank lines to 2.
• Remove spaces before punctuation.
• Preserve content inside fenced code blocks unchanged.

11. Do not use innerText for whole message containers as the primary extraction method.

    • It may be used only as a fallback when DOM parsing fails.

12. Add small local test fixtures or comments for:

• paragraph with bold inline phrases
• paragraph followed by unordered list
• paragraph followed by numbered list
• nested list if easy
• code block
• quote/em dash sentence

Suggested helper functions:

• nodeToMarkdown(node, context)
• childrenToInlineMarkdown(node, context)
• blockChildrenToMarkdown(node, context)
• listToMarkdown(listNode, context)
• listItemToMarkdown(liNode, context)
• codeBlockToMarkdown(preNode)
• cleanupMarkdownOutsideCodeFences(markdown)

Acceptance criteria:

• The sample assistant response preserves paragraphs.
• Catastrophe examples become a Markdown bullet list.
• The “important distinctions” examples become separate paragraphs or list items if ChatGPT rendered them as list items.
• Bold phrases remain inline, not on separate lines.
• Periods and commas remain attached to the correct sentence.
• Code blocks remain intact.
• Message order and role detection are unchanged.
• No LLM calls.
• No Project Thoth application dependency.