7.7 KiB
adr, title, status, date, authors
| adr | title | status | date | authors | |
|---|---|---|---|---|---|
| 003 | Capture Connector Architecture | Accepted | 2026-07-09 |
|
ADR-003: Capture Connector Architecture
Status
Accepted
Context
Project Thoth is intended to preserve knowledge from a growing ecosystem of AI assistants and digital systems.
Initially, the first connector targeted ChatGPT. During development it became clear that the ChatGPT connector was not unique. Although each platform exposes different APIs and DOM structures, every connector ultimately performs the same high-level task:
- Capture source material.
- Transform it into a canonical representation.
- Deliver the result to Project Thoth.
The platform-specific logic lies almost entirely in how the source material is discovered. Once discovered, the remainder of the processing pipeline is largely identical.
Rather than implementing each connector as a monolithic application, Project Thoth should define a common connector architecture with clearly defined extension points.
Decision
All Project Thoth Capture Connectors SHALL implement the same logical architecture.
Source Platform
│
▼
Platform Discovery Layer
│
▼
Canonical Conversation Model
│
▼
Platform-Neutral Transformation
│
▼
Canonical conversation.md
│
▼
Project Thoth Pipeline
Only the Discovery Layer is expected to be platform-specific.
Connector Responsibilities
Capture Connectors are responsible only for preserving source material.
They SHALL:
- Capture conversations
- Capture attachments
- Preserve ordering
- Preserve formatting where practical
- Preserve metadata supplied by the source platform
- Produce canonical Project Thoth documents
They SHALL NOT:
- Summarize
- Classify
- Generate tags
- Generate YAML metadata
- Generate manifests
- Generate harvests
- Perform semantic analysis
- Invoke LLMs
- Modify user content
Connectors are intentionally "dumb."
Their responsibility is faithful preservation.
Canonical Connector Pipeline
Every connector SHALL implement the following stages.
Preprocessing (Optional)
│
▼
Discovery
│
▼
Intermediate Representation
│
▼
Transformation
│
▼
Serialization
│
▼
Output
Stage 0 — Preprocessing (Optional)
Purpose:
Prepare the source for capture.
Examples:
- Render virtualized content
- Expand collapsed sections
- Load lazy content
- Wait for streaming responses to complete
Preprocessing SHALL NOT modify user content.
Stage 1 — Discovery
Purpose:
Locate the logical content exposed by the source platform.
Responsibilities include:
- Locate conversation root
- Locate conversation turns
- Determine ordering
- Determine speaker
- Detect unsupported content
- Detect partial rendering
Discovery is platform-specific.
Stage 2 — Intermediate Representation
Purpose:
Represent captured information in a platform-neutral model.
Example:
interface Conversation {
sourcePlatform: string;
title: string;
url: string;
capturedAt: Date;
turns: ConversationTurn[];
}
interface ConversationTurn {
turnIndex: number;
role: string;
captureStatus: string;
content: DocumentFragment | HTMLElement | string;
}
The Intermediate Representation (IR) is the contract between Discovery and Transformation.
Stage 3 — Transformation
Purpose:
Convert platform-specific content into canonical Project Thoth Markdown.
Responsibilities include:
- HTML → Markdown
- Paragraph preservation
- Lists
- Tables
- Links
- Images
- Code blocks
- Inline formatting
Transformation is expected to be reusable across platforms.
Stage 4 — Serialization
Purpose:
Generate the canonical output artifacts.
Current artifacts include:
conversation.md
Future artifacts may include:
- Attachments
- Asset manifests
- Conversation package formats
Serialization SHALL NOT reinterpret content.
Platform Independence
The connector architecture intentionally separates platform-specific logic from platform-neutral logic.
Examples:
| Component | Platform Specific |
|---|---|
| Discovery | Yes |
| Preprocessing | Mostly |
| Transformation | No |
| Serialization | No |
This minimizes duplication across connectors.
Supported Connector Types
The architecture is intended to support connectors including, but not limited to:
- ChatGPT
- Claude
- Gemini
- Microsoft Copilot
- Open WebUI
- Perplexity
- Cursor
- GitHub Copilot Chat
- Future browser-based AI assistants
Additional connectors should primarily require implementation of Discovery and, where necessary, Preprocessing.
Design Principles
Fidelity Over Intelligence
Connectors preserve information.
They do not interpret information.
Platform Neutrality
Internal Project Thoth formats are independent of any external platform.
No downstream component should need to know whether content originated from ChatGPT, Gemini, Claude, or another system.
Composability
Each stage should be independently testable and replaceable.
This enables improvements to one stage without affecting others.
Deterministic Output
Running the connector multiple times against the same rendered conversation should produce equivalent output.
Fail Gracefully
When unsupported content is encountered:
- Preserve placeholders
- Preserve ordering
- Record warnings
- Never silently discard content
Error Handling
Capture is considered successful when source material is faithfully preserved.
If content cannot be rendered or extracted:
- Report the issue
- Preserve available context
- Continue processing remaining content
Partial capture is preferred over silent failure.
Consequences
Positive
- Uniform architecture across all connectors
- Reduced duplication
- Easier testing
- Improved maintainability
- Simplified onboarding for new connector development
- Reusable transformation and serialization components
Negative
- Additional abstraction layers
- Requires maintenance of a shared Intermediate Representation
- Slightly higher initial implementation effort
Alternatives Considered
Monolithic Platform Connectors
Rejected.
Embedding discovery, transformation, and serialization into a single implementation results in duplicated logic and inconsistent behavior across connectors.
Direct Platform-to-Markdown Conversion
Rejected.
Coupling discovery with rendering makes debugging difficult and limits reuse.
Relationship to Other ADRs
- ADR-001 establishes the overall Project Thoth repository and project structure.
- ADR-002 defines the canonical capture pipeline used within connectors.
- ADR-003 defines the architectural responsibilities, lifecycle, and composition of Capture Connectors as reusable platform adapters.
Together, these ADRs establish the foundation for a connector ecosystem rather than a collection of independent integrations.
Future Considerations
Future enhancements may include:
- Native API-based connectors where supported
- Hybrid API + browser capture
- Incremental conversation synchronization
- Background monitoring of supported platforms
- Signed connector packages
- Connector capability negotiation
- Automated regression testing against captured DOM snapshots
The architectural principles defined in this ADR are expected to remain stable even as individual source platforms evolve.