47 lines
2.1 KiB
Markdown
47 lines
2.1 KiB
Markdown
Fix Task 3A conversation discovery over-capture and duplication.
|
|
|
|
Problem:
|
|
The extractor is now capturing the ChatGPT application shell, sidebar, navigation, composer UI, and duplicate parent/child containers. It also captures large flattened blocks containing both "You said" and "ChatGPT said" instead of discrete messages.
|
|
|
|
Goal:
|
|
Discover only the conversation transcript stream and return one message object per actual conversation turn.
|
|
|
|
Requirements:
|
|
1. Exclude ChatGPT app shell:
|
|
- Skip sidebar/nav/history/project/account areas.
|
|
- Skip composer/input area.
|
|
- Skip buttons, menus, share controls, footer text, scripts, and hidden UI.
|
|
|
|
2. Scope discovery to the main conversation region:
|
|
- Prefer main[role="main"] or the primary conversation scroll container.
|
|
- Do not search the entire document body unless used only as a fallback.
|
|
|
|
3. Deduplicate containers:
|
|
- Do not capture both a parent and its child if both contain the same message text.
|
|
- Prefer the smallest stable container that represents one message.
|
|
- Track DOM containment and normalized text fingerprints to remove duplicates.
|
|
|
|
4. Split transcript turns:
|
|
- Do not allow one captured block to contain multiple "You said:" / "ChatGPT said:" turns.
|
|
- Each actual user or assistant turn should become a separate message.
|
|
|
|
5. Keep fallback behavior, but only after filtering:
|
|
- Unknown containers are allowed, but they must come from the conversation region, not the app shell.
|
|
- Unknown should mean unknown conversation content, not random ChatGPT UI.
|
|
|
|
6. Add debug counts:
|
|
- candidateContainersFound
|
|
- afterShellFilter
|
|
- afterDeduplication
|
|
- messagesReturned
|
|
- skippedShellContainers
|
|
- skippedDuplicateContainers
|
|
|
|
Acceptance criteria:
|
|
- No "Skip to content" block.
|
|
- No sidebar chat history.
|
|
- No repeated full transcript blocks.
|
|
- No window.__oai_logHTML or script text.
|
|
- Actual user/assistant turns are separate messages.
|
|
- Image generation/edit placeholders may remain as unknown or unsupported, but only once.
|
|
- Completeness remains favored, but not by capturing the whole app shell. |