Project-Thoth/applications/chatgpt-capture/codex-work-orders/debugging/0005-implementation3a-over-capture-defect.md

Fix Task 3A conversation discovery over-capture and duplication.

Problem: The extractor is now capturing the ChatGPT application shell, sidebar, navigation, composer UI, and duplicate parent/child containers. It also captures large flattened blocks containing both "You said" and "ChatGPT said" instead of discrete messages.

Goal: Discover only the conversation transcript stream and return one message object per actual conversation turn.

Requirements:

1. Exclude ChatGPT app shell:

   • Skip sidebar/nav/history/project/account areas.
   • Skip composer/input area.
   • Skip buttons, menus, share controls, footer text, scripts, and hidden UI.

2. Scope discovery to the main conversation region:

   • Prefer main[role="main"] or the primary conversation scroll container.
   • Do not search the entire document body unless used only as a fallback.

3. Deduplicate containers:

   • Do not capture both a parent and its child if both contain the same message text.
   • Prefer the smallest stable container that represents one message.
   • Track DOM containment and normalized text fingerprints to remove duplicates.

4. Split transcript turns:

   • Do not allow one captured block to contain multiple "You said:" / "ChatGPT said:" turns.
   • Each actual user or assistant turn should become a separate message.

5. Keep fallback behavior, but only after filtering:

   • Unknown containers are allowed, but they must come from the conversation region, not the app shell.
   • Unknown should mean unknown conversation content, not random ChatGPT UI.

6. Add debug counts:

   • candidateContainersFound
   • afterShellFilter
   • afterDeduplication
   • messagesReturned
   • skippedShellContainers
   • skippedDuplicateContainers

Acceptance criteria:

• No "Skip to content" block.
• No sidebar chat history.
• No repeated full transcript blocks.
• No window.__oai_logHTML or script text.
• Actual user/assistant turns are separate messages.
• Image generation/edit placeholders may remain as unknown or unsupported, but only once.
• Completeness remains favored, but not by capturing the whole app shell.