6.3 KiB
Project Thoth Architecture
Version: 1.0
Status: Living Document
Purpose
This document describes the canonical architecture of Project Thoth.
It is intended to answer a single question:
How does information flow through the Project Thoth ecosystem?
This document describes architecture, not implementation.
Current implementation status is maintained separately in .thoth/00-project-state.md.
Architectural Principles
Project Thoth is built upon the following principles:
- Source artifacts are permanent.
- Generated artifacts are disposable.
- Every transformation is deterministic.
- AI providers are interchangeable.
- Markdown is the canonical document format.
- Knowledge is owned by the archive, not the AI.
- Processing stages should be loosely coupled.
- Every stage should be independently testable.
High-Level Architecture
External Sources
│
▼
Capture Connectors
│
▼
Canonical Source Archive
│
▼
Processing Pipeline
│
┌──────────────┼──────────────┐
▼ ▼ ▼
Metadata Manifest Harvests
Generation Generation Generation
│ │ │
└──────────────┼──────────────┘
▼
Knowledge Repository
│
▼
Search / Retrieval Layer
│
▼
AI Consumers
Layers
1. External Sources
Project Thoth captures knowledge from external systems.
Examples include:
- ChatGPT
- Claude
- Gemini
- Microsoft Copilot
- Open WebUI
- Local LLMs
- Documentation systems
- Markdown repositories
- Future knowledge sources
These systems are considered transient.
They are never considered the canonical repository.
2. Capture Connectors
Capture Connectors preserve source material.
Responsibilities include:
- Capture conversations
- Preserve ordering
- Preserve formatting
- Preserve attachments
- Produce canonical source documents
Capture Connectors do not:
- Summarize
- Interpret
- Classify
- Generate metadata
- Perform AI reasoning
They are intentionally deterministic.
3. Canonical Source Archive
The archive contains the preserved source material.
Typical artifacts include:
- conversation.md
- source assets
- attachments
The archive is immutable.
Once source material has been captured it should not be modified.
If processing changes in the future, artifacts are regenerated from the archived source.
4. Processing Pipeline
Processors transform source artifacts into derived artifacts.
Typical processors include:
- Metadata generation
- Manifest generation
- Harvest generation
- Search indexing
- Embedding generation
Processors never modify source artifacts.
They only generate derived artifacts.
5. Knowledge Repository
The repository contains:
- Source artifacts
- Derived artifacts
- Metadata
- Relationships
This becomes the canonical knowledge base for Project Thoth.
6. Consumers
Consumers interact with the repository.
Examples:
- Local LLMs
- Search engines
- Knowledge assistants
- Documentation systems
- Human readers
Consumers should never depend on the original AI platform.
Capture Connector Architecture
Every Capture Connector implements the same logical pipeline.
Optional Preprocessing
│
▼
Conversation Discovery
│
▼
Intermediate Representation
│
▼
Content Transformation
│
▼
Markdown Serialization
│
▼
conversation.md
This architecture is defined by ADR-002.
Only the Discovery stage is expected to be platform-specific.
Processing Philosophy
Project Thoth separates preservation from interpretation.
Capture Once
↓
Preserve Forever
↓
Process Many Times
This allows improvements to processors without requiring the original source to be recaptured.
Canonical Artifacts
Project Thoth distinguishes between canonical artifacts and generated artifacts.
Canonical
Examples:
- conversation.md
- Source attachments
- Images
- Audio
- Video
Canonical artifacts are permanent.
Generated
Examples:
- Metadata
- Manifests
- Harvests
- Embeddings
- Search indexes
Generated artifacts may be deleted and regenerated at any time.
Repository Structure
A typical Project Thoth repository contains:
.thoth/
Project self-description
applications/
Executable applications
processors/
Artifact processors
specifications/
Canonical document specifications
docs/
Architecture and ADRs
codex/
Engineering work orders
archive/
Captured source material
generated/
Derived artifacts
Implementation details may evolve while preserving the architectural boundaries described in this document.
Architectural Boundaries
Each layer has a single responsibility.
| Layer | Responsibility |
|---|---|
| Capture Connectors | Preserve source material |
| Archive | Store canonical source |
| Processors | Generate derived artifacts |
| Repository | Organize knowledge |
| Consumers | Read knowledge |
No layer should assume the responsibilities of another.
Design Goals
The architecture is intended to achieve:
- Long-term preservation
- Platform independence
- Deterministic processing
- Reproducibility
- Testability
- Portability
- AI interoperability
Relationship to Other Documents
This document defines how Project Thoth is organized.
Related documents include:
- 00-project-state.md — Current implementation status.
- 20-philosophy.md — Guiding principles.
- 30-glossary.md — Canonical terminology.
- 40-coding-standards.md — Development standards.
- ADR-001 — Project structure.
- ADR-002 — Capture connector pipeline.
- ADR-003 — Capture connector architecture.
Together these documents form the canonical self-description of the project.