---
adr: 003
title: Capture Connector Architecture
status: Accepted
date: 2026-07-09
authors:
  - Ken Schaefer
---

# ADR-003: Capture Connector Architecture

## Status

Accepted

---

# Context

Project Thoth is intended to preserve knowledge from a growing ecosystem of AI assistants and digital systems.

Initially, the first connector targeted ChatGPT. During development it became clear that the ChatGPT connector was not unique. Although each platform exposes different APIs and DOM structures, every connector ultimately performs the same high-level task:

1. Capture source material.
2. Transform it into a canonical representation.
3. Deliver the result to Project Thoth.

The platform-specific logic lies almost entirely in *how the source material is discovered*. Once discovered, the remainder of the processing pipeline is largely identical.

Rather than implementing each connector as a monolithic application, Project Thoth should define a common connector architecture with clearly defined extension points.

---

# Decision

All Project Thoth Capture Connectors SHALL implement the same logical architecture.

```
                 Source Platform
                        │
                        ▼
             Platform Discovery Layer
                        │
                        ▼
          Canonical Conversation Model
                        │
                        ▼
         Platform-Neutral Transformation
                        │
                        ▼
          Canonical conversation.md
                        │
                        ▼
              Project Thoth Pipeline
```

Only the Discovery Layer is expected to be platform-specific.

---

# Connector Responsibilities

Capture Connectors are responsible only for preserving source material.

They SHALL:

- Capture conversations
- Capture attachments
- Preserve ordering
- Preserve formatting where practical
- Preserve metadata supplied by the source platform
- Produce canonical Project Thoth documents

They SHALL NOT:

- Summarize
- Classify
- Generate tags
- Generate YAML metadata
- Generate manifests
- Generate harvests
- Perform semantic analysis
- Invoke LLMs
- Modify user content

Connectors are intentionally "dumb."

Their responsibility is faithful preservation.

---

# Canonical Connector Pipeline

Every connector SHALL implement the following stages.

```
Preprocessing (Optional)
        │
        ▼
Discovery
        │
        ▼
Intermediate Representation
        │
        ▼
Transformation
        │
        ▼
Serialization
        │
        ▼
Output
```

---

# Stage 0 — Preprocessing (Optional)

Purpose:

Prepare the source for capture.

Examples:

- Render virtualized content
- Expand collapsed sections
- Load lazy content
- Wait for streaming responses to complete

Preprocessing SHALL NOT modify user content.

---

# Stage 1 — Discovery

Purpose:

Locate the logical content exposed by the source platform.

Responsibilities include:

- Locate conversation root
- Locate conversation turns
- Determine ordering
- Determine speaker
- Detect unsupported content
- Detect partial rendering

Discovery is platform-specific.

---

# Stage 2 — Intermediate Representation

Purpose:

Represent captured information in a platform-neutral model.

Example:

```typescript
interface Conversation {

    sourcePlatform: string;

    title: string;

    url: string;

    capturedAt: Date;

    turns: ConversationTurn[];
}

interface ConversationTurn {

    turnIndex: number;

    role: string;

    captureStatus: string;

    content: DocumentFragment | HTMLElement | string;
}
```

The Intermediate Representation (IR) is the contract between Discovery and Transformation.

---

# Stage 3 — Transformation

Purpose:

Convert platform-specific content into canonical Project Thoth Markdown.

Responsibilities include:

- HTML → Markdown
- Paragraph preservation
- Lists
- Tables
- Links
- Images
- Code blocks
- Inline formatting

Transformation is expected to be reusable across platforms.

---

# Stage 4 — Serialization

Purpose:

Generate the canonical output artifacts.

Current artifacts include:

- `conversation.md`

Future artifacts may include:

- Attachments
- Asset manifests
- Conversation package formats

Serialization SHALL NOT reinterpret content.

---

# Platform Independence

The connector architecture intentionally separates platform-specific logic from platform-neutral logic.

Examples:

| Component | Platform Specific |
|-----------|-------------------|
| Discovery | Yes |
| Preprocessing | Mostly |
| Transformation | No |
| Serialization | No |

This minimizes duplication across connectors.

---

# Supported Connector Types

The architecture is intended to support connectors including, but not limited to:

- ChatGPT
- Claude
- Gemini
- Microsoft Copilot
- Open WebUI
- Perplexity
- Cursor
- GitHub Copilot Chat
- Future browser-based AI assistants

Additional connectors should primarily require implementation of Discovery and, where necessary, Preprocessing.

---

# Design Principles

## Fidelity Over Intelligence

Connectors preserve information.

They do not interpret information.

---

## Platform Neutrality

Internal Project Thoth formats are independent of any external platform.

No downstream component should need to know whether content originated from ChatGPT, Gemini, Claude, or another system.

---

## Composability

Each stage should be independently testable and replaceable.

This enables improvements to one stage without affecting others.

---

## Deterministic Output

Running the connector multiple times against the same rendered conversation should produce equivalent output.

---

## Fail Gracefully

When unsupported content is encountered:

- Preserve placeholders
- Preserve ordering
- Record warnings
- Never silently discard content

---

# Error Handling

Capture is considered successful when source material is faithfully preserved.

If content cannot be rendered or extracted:

- Report the issue
- Preserve available context
- Continue processing remaining content

Partial capture is preferred over silent failure.

---

# Consequences

## Positive

- Uniform architecture across all connectors
- Reduced duplication
- Easier testing
- Improved maintainability
- Simplified onboarding for new connector development
- Reusable transformation and serialization components

## Negative

- Additional abstraction layers
- Requires maintenance of a shared Intermediate Representation
- Slightly higher initial implementation effort

---

# Alternatives Considered

## Monolithic Platform Connectors

Rejected.

Embedding discovery, transformation, and serialization into a single implementation results in duplicated logic and inconsistent behavior across connectors.

---

## Direct Platform-to-Markdown Conversion

Rejected.

Coupling discovery with rendering makes debugging difficult and limits reuse.

---

# Relationship to Other ADRs

- **ADR-001** establishes the overall Project Thoth repository and project structure.
- **ADR-002** defines the canonical capture pipeline used within connectors.
- **ADR-003** defines the architectural responsibilities, lifecycle, and composition of Capture Connectors as reusable platform adapters.

Together, these ADRs establish the foundation for a connector ecosystem rather than a collection of independent integrations.

---

# Future Considerations

Future enhancements may include:

- Native API-based connectors where supported
- Hybrid API + browser capture
- Incremental conversation synchronization
- Background monitoring of supported platforms
- Signed connector packages
- Connector capability negotiation
- Automated regression testing against captured DOM snapshots

The architectural principles defined in this ADR are expected to remain stable even as individual source platforms evolve.