kenschae/thoth-gateway
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
master
thoth-gateway/.thoth/50-roadmap.md
T

22 KiB
Raw Permalink Blame History

Thoth Gateway Infrastructure Roadmap

Version: 0.1.0 Status: Living Document Last Updated: 2026-07-28

Purpose

This roadmap describes the intended evolution of the Thoth Gateway infrastructure.

Unlike 00-project-state.md, which records the current implementation state, this document defines the long-term direction of the platform.

The roadmap is intentionally architectural and high-level.

Implementation details belong in:

  • Work Orders.
  • Architecture Decision Records.
  • Service documentation.
  • Deployment definitions.
  • Operational procedures.

Vision

The Thoth Gateway infrastructure will become the governed service layer between:

  • Human users.
  • Project Thoth knowledge.
  • Local AI systems.
  • Frontier AI providers.
  • Agentic tools.
  • External applications.

Its primary objective is to make knowledge and intelligence accessible without allowing any interface, model, runtime, or provider to become the owner of the system.

Gateway should support a durable environment in which:

  • Services are reproducible.
  • Knowledge remains canonical.
  • Access is governed.
  • Egress is controlled.
  • Local inference is available.
  • Frontier inference is policy-driven.
  • Agent execution is constrained.
  • Recovery does not depend on memory.
  • Infrastructure can evolve without losing architectural understanding.

Guiding Principles

The roadmap follows these principles:

  • Build vertically before horizontally.
  • Establish one complete workflow before adding parallel services.
  • Preserve architecture before expanding implementation.
  • Keep Gateway orchestration separate from Oracle inference.
  • Keep knowledge separate from application state.
  • Introduce retrieval only with authorization boundaries.
  • Introduce frontier providers only with egress policy.
  • Introduce Agents only with delegated authority.
  • Prefer deterministic systems where practical.
  • Favor replaceable services and stable interfaces.
  • Make every milestone independently useful.
  • Treat documentation and recovery as deliverables.
  • Avoid adding infrastructure merely because a tool is available.

Phase 1 — Gateway Foundation

Objective

Establish thoth-gateway as a stable, documented, and reproducible service host.

Deliverables

  • Rebuilt Gateway operating system.
  • Canonical hostname and network identity.
  • Administrative SSH access.
  • Stable storage layout.
  • Docker Engine.
  • Docker Compose plugin.
  • Non-root Docker administration.
  • Infrastructure Git repository.
  • .thoth project self-description.
  • Repository standards.
  • Base operational documentation.
  • Initial verification scripts.

Success Criteria

  • Gateway survives reboot with the expected configuration.
  • Docker starts automatically.
  • The administrative user can operate Docker without sudo.
  • The repository describes the system without relying on conversation history.
  • Deployment definitions are separated from runtime data.
  • The host can be rebuilt from documented procedures.

Phase 2 — Local Inference Interface

Objective

Establish one complete user-to-local-model workflow.

Deliverables

  • Open WebUI Service Project.
  • Persistent Open WebUI data.
  • Oracle endpoint configuration.
  • Model discovery.
  • Gemma 4 availability.
  • Direct inference verification.
  • Browser-based end-to-end test.
  • Open WebUI service documentation.
  • Backup and recovery scope.

Initial Workflow

Browser
    ↓
Open WebUI on Gateway
    ↓
Ollama on Oracle
    ↓
Gemma 4
    ↓
Open WebUI
    ↓
Browser

Success Criteria

  • Open WebUI operates persistently on Gateway.
  • Gateway can enumerate Oracle models.
  • Gemma 4 can be selected in Open WebUI.
  • A prompt submitted through Open WebUI is processed on Oracle.
  • The response returns successfully to the browser.
  • Conversation state survives service restart.
  • Deployment and verification procedures are documented.

Phase 3 — Operational Baseline

Objective

Make the initial Gateway services operable beyond first deployment.

Deliverables

  • Service health checks.
  • Host verification scripts.
  • Dependency checks.
  • Version reporting.
  • Log access procedures.
  • Backup procedures.
  • Restore procedures.
  • Upgrade procedures.
  • Rollback guidance.
  • Configuration validation.
  • Operational runbooks.

Success Criteria

  • Service failure can be detected.
  • Oracle dependency failure can be distinguished from Open WebUI failure.
  • Runtime data can be backed up.
  • A service can be restored from documented steps.
  • Upgrades can be performed deliberately.
  • Operational knowledge is stored in the repository.

Phase 4 — Internal Service Ingress

Objective

Provide stable, controlled access to Gateway-hosted services.

Deliverables

  • Internal DNS names.
  • Reverse proxy.
  • TLS for internal services.
  • Canonical service URLs.
  • Reduced direct port exposure.
  • Documented ingress rules.
  • Session and authentication review.
  • Firewall alignment.

Candidate Service Names

chat.witheredsanctum.com
gateway.witheredsanctum.com

Final naming requires an Architecture Decision Record.

Success Criteria

  • Users access services through stable names.
  • TLS is consistently applied.
  • Direct container ports are limited to required administrative use.
  • Internal and external exposure are clearly distinguished.
  • Ingress configuration is reproducible and documented.

Phase 5 — Open WebUI Capture

Objective

Preserve Open WebUI conversations as canonical Project Thoth source artifacts.

Deliverables

  • Open WebUI Capture Connector.
  • Conversation discovery.
  • Conversation Turn preservation.
  • Attachment handling.
  • Markdown serialization.
  • Source metadata.
  • Conversation manifests.
  • Capture verification.
  • Archive destination rules.

Intended Flow

Open WebUI Conversation
    ↓
Open WebUI Capture Connector
    ↓
Canonical conversation.md
    ↓
Project Thoth Archive
    ↓
Metadata and Manifest Processing

Success Criteria

  • Open WebUI conversations can be preserved outside Open WebUI.
  • Preserved artifacts remain readable without Open WebUI.
  • Conversation ordering and context are retained.
  • Application deletion does not destroy captured knowledge.
  • Generated metadata can be recreated from canonical artifacts.

Phase 6 — Deterministic Retrieval Foundation

Objective

Provide reliable retrieval across Project Thoth knowledge without requiring AI reasoning.

Deliverables

  • Full-text search.
  • Metadata search.
  • Collection search.
  • Source address resolution.
  • Artifact relationship navigation.
  • Citation generation.
  • Search result ranking.
  • Retrieval interfaces.
  • Representative test fixtures.

Success Criteria

  • Users can locate knowledge without a language model.
  • Search results retain links to canonical artifacts.
  • Retrieval works independently of Open WebUI.
  • Retrieval remains usable if the current model provider is replaced.
  • Search behavior can be tested and reproduced.

Phase 7 — Identity and Authorization

Objective

Ensure that retrieval and service access respect user identity and source permissions.

Deliverables

  • Identity model.
  • Authentication integration.
  • Access Policy model.
  • Collection permissions.
  • Artifact-level permissions.
  • Source-system permission mapping.
  • Authorization service.
  • Permission-aware retrieval.
  • Audit Events.
  • Denial behavior.
  • Test identities and permission fixtures.

Intended Flow

Authenticated User
    ↓
Search Request
    ↓
Candidate Results
    ↓
Access Policy Evaluation
    ↓
Authorized Results
    ↓
Source Resolution

Success Criteria

  • Search does not disclose unauthorized content.
  • Source Resolution occurs only after authorization.
  • A user cannot access an artifact merely because it is indexed.
  • Authorization decisions are auditable.
  • Language models do not determine access rights.
  • Permission failures are explicit and testable.

Phase 8 — Governed Context Packaging

Objective

Prepare authorized knowledge for AI Consumers without losing source identity or policy constraints.

Deliverables

  • Context Package specification.
  • Context Packager.
  • Token-budget handling.
  • Source excerpt selection.
  • Citation preservation.
  • Metadata inclusion rules.
  • Data classification.
  • Consumer-specific formatting.
  • Prompt-injection separation.
  • Context Package tests.

Success Criteria

  • Context includes only authorized material.
  • Source references remain intact.
  • Retrieved content is clearly separated from system instructions.
  • Context packaging is independent of a specific model.
  • The same approved sources can be packaged for different Consumers.
  • Restricted information is not silently included.

Phase 9 — Provider Abstraction

Objective

Allow Gateway services to use multiple AI providers through stable interfaces.

Deliverables

  • Local Provider Interface.
  • Frontier Provider Interface.
  • Provider capability metadata.
  • Provider health checks.
  • Model availability checks.
  • Provider configuration.
  • Provider-specific adapters.
  • Fallback policy.
  • Failure reporting.
  • Provider test harness.

Initial Providers

  • Oracle through Ollama.
  • Approved OpenAI services.
  • Future local runtimes.
  • Future frontier providers.

Success Criteria

  • Consumers do not hard-code Ollama-specific behavior unnecessarily.
  • A model or provider can be replaced without restructuring the archive.
  • Provider failure is distinguishable from retrieval or policy failure.
  • Provider availability can be tested independently.
  • Provider selection can be governed centrally.

Phase 10 — Egress Policy and Routing

Objective

Determine whether information may be sent outside the local environment and which provider may receive it.

Deliverables

  • Data classification model.
  • Egress Policy.
  • Provider approval rules.
  • Local-only classifications.
  • De-identification workflow.
  • Redaction or summarization controls.
  • Provider Routing.
  • Egress audit records.
  • Denial and fallback behavior.
  • Policy test fixtures.

Intended Flow

Authorized Context
    ↓
Data Classification
    ↓
Egress Policy
    ↓
Provider Routing
    ├── Local Provider
    └── Approved Frontier Provider

Success Criteria

  • Restricted information remains local.
  • Frontier use is explicit and auditable.
  • Provider selection is based on policy, not interface convenience.
  • De-identification occurs before egress when required.
  • A denied frontier request can fall back safely to local inference where appropriate.
  • Models do not override provider policy.

Phase 11 — Knowledge-Augmented Inference

Objective

Combine governed retrieval, context packaging, and provider routing into one complete workflow.

Deliverables

  • Knowledge query interface.
  • Permission-aware retrieval.
  • Context Package creation.
  • Provider selection.
  • Model request.
  • Source citations.
  • Response traceability.
  • Failure handling.
  • End-to-end tests.
  • Open WebUI integration.

Intended Workflow

User Request
    ↓
Identity
    ↓
Search
    ↓
Authorization
    ↓
Source Resolution
    ↓
Context Packaging
    ↓
Egress Policy
    ↓
Provider Routing
    ↓
Inference
    ↓
Response with Source References

Success Criteria

  • Responses are grounded in authorized Project Thoth knowledge.
  • Sources remain identifiable.
  • Unauthorized content does not enter the Context Package.
  • Local-only information does not reach frontier providers.
  • Retrieval, policy, and inference failures are distinguishable.
  • The workflow is testable end to end.

Phase 12 — Agent Runtime

Objective

Enable governed multi-step work through Agents.

Deliverables

  • Agent Runtime.
  • Task model.
  • Delegated Authority model.
  • Tool registry.
  • Tool permission model.
  • Execution limits.
  • Approval checkpoints.
  • Agent state handling.
  • Audit Events.
  • Failure and recovery behavior.
  • Agent Work Order format.
  • Agent execution tests.

Success Criteria

  • Agents operate only within explicit authority.
  • Tool access is granted per task.
  • Knowledge scope is constrained.
  • Provider restrictions are enforced.
  • Sensitive actions can require human approval.
  • Agent work produces reviewable evidence.
  • Model output alone cannot grant new authority.

Phase 13 — Tool and Integration Framework

Objective

Provide standardized, governed interfaces to external systems.

Candidate Integrations

  • Project Thoth archive.
  • Nextcloud.
  • Git repositories.
  • Email.
  • Calendar.
  • Document repositories.
  • Local file systems.
  • OpenEMR.
  • Future organizational systems.

Deliverables

  • Tool interface specification.
  • Credential scoping.
  • Read and write separation.
  • Integration health checks.
  • Input validation.
  • Audit logging.
  • Tool result normalization.
  • Permission mapping.
  • Agent-safe interfaces.
  • Integration test harness.

Success Criteria

  • Integrations do not expose unrestricted credentials.
  • Read and write permissions are distinguishable.
  • Tool output is treated as untrusted input.
  • Actions are attributable to a user or Agent.
  • Integration failures do not corrupt task state.
  • New integrations can be added through stable interfaces.

Phase 14 — Secrets Management

Objective

Replace ad hoc secret storage with a managed, auditable mechanism.

Deliverables

  • Secret inventory.
  • Secret ownership model.
  • Secret injection mechanism.
  • Rotation procedures.
  • Access controls.
  • Backup and recovery policy.
  • Agent credential restrictions.
  • Audit Events.
  • Migration from local .env secrets.
  • Emergency revocation procedure.

Success Criteria

  • Secrets are not stored in Git.
  • Services receive only required secrets.
  • Secret rotation does not require architectural changes.
  • Agent access to credentials is explicit.
  • Compromised secrets can be revoked quickly.
  • Recovery procedures do not expose secrets unnecessarily.

Phase 15 — Monitoring and Operational Intelligence

Objective

Provide actionable visibility into Gateway, Oracle, services, and workflows.

Deliverables

  • Host monitoring.
  • Container monitoring.
  • Endpoint monitoring.
  • Oracle availability monitoring.
  • Model availability monitoring.
  • Storage monitoring.
  • Backup monitoring.
  • Workflow health checks.
  • Alerting.
  • Operational dashboards.
  • Retention rules.

Success Criteria

  • Failures are detected before users must diagnose them manually.
  • Alerts identify the failing layer.
  • Monitoring distinguishes host, container, service, dependency, and workflow failures.
  • Sensitive content is not unnecessarily logged.
  • Operational data supports recovery and capacity planning.

Phase 16 — Backup and Disaster Recovery

Objective

Create tested recovery paths for Gateway services and Project Thoth integrations.

Deliverables

  • Service backup schedules.
  • Backup retention.
  • Backup encryption.
  • Off-host backup storage.
  • Restore automation.
  • Recovery runbooks.
  • Recovery testing.
  • Recovery evidence.
  • Host rebuild procedure.
  • Oracle dependency recovery procedure.
  • Service priority classification.

Success Criteria

  • Gateway can be rebuilt from documented artifacts.
  • Critical Service Data can be restored.
  • Recovery procedures are tested.
  • Recovery time and data-loss expectations are documented.
  • Backup success is verified rather than assumed.
  • Infrastructure and knowledge backups remain distinct.

Phase 17 — High Availability and Capacity Expansion

Objective

Increase resilience and capacity only after operational requirements justify it.

Potential Capabilities

  • Additional inference GPUs.
  • Additional Oracle nodes.
  • Provider load balancing.
  • Service redundancy.
  • Replicated storage.
  • Failover endpoints.
  • Queue-based execution.
  • Distributed Agent workers.
  • Capacity-aware Provider Routing.

Success Criteria

  • Expansion solves measured constraints.
  • Added complexity is justified by operational evidence.
  • Failure behavior is documented.
  • Service interfaces remain stable.
  • Knowledge remains independent of the expanded infrastructure.

Phase 18 — Project Thoth Operations Platform

Objective

Transform Gateway from a collection of services into a coherent knowledge-operations platform.

Potential Capabilities

  • Unified administration interface.
  • Collection management.
  • Policy administration.
  • Provider administration.
  • Agent administration.
  • Capture status.
  • Processing status.
  • Knowledge health metrics.
  • Search diagnostics.
  • Audit review.
  • Backup and recovery status.
  • Workflow scheduling.

Success Criteria

  • Operators can understand system state from one coherent operational model.
  • Administrative interfaces reflect canonical architecture.
  • The platform does not hide critical configuration or policy in opaque state.
  • Services remain independently replaceable.
  • Project Thoth knowledge remains portable.

Parallel Work Streams

Several work streams evolve across multiple phases.

Documentation

Includes:

  • .thoth self-description.
  • Service READMEs.
  • Architecture Decision Records.
  • Work Orders.
  • Runbooks.
  • Recovery procedures.
  • Network maps.
  • Security documentation.

Documentation is part of implementation, not post-implementation cleanup.

Security

Includes:

  • Authentication.
  • Authorization.
  • Network exposure.
  • Secret handling.
  • Container hardening.
  • Prompt-injection boundaries.
  • Agent permissions.
  • Egress controls.
  • Audit Events.

Security decisions should develop alongside each capability.

Testing

Includes:

  • Static validation.
  • Host tests.
  • Service tests.
  • Integration tests.
  • End-to-End Tests.
  • Policy tests.
  • Permission tests.
  • Backup tests.
  • Recovery tests.
  • Regression fixtures.

No phase is complete solely because its components start.

Capture Connectors

Potential Gateway-related Capture Connectors include:

  • Open WebUI.
  • Agent interactions.
  • Email.
  • Document systems.
  • Git and development tools.
  • Future AI interfaces.

Capture remains separate from interpretation.

Provider Integration

Potential providers include:

  • Ollama.
  • Future local model runtimes.
  • OpenAI.
  • Google.
  • Anthropic.
  • Specialized local models.
  • Specialized cloud models.

Providers are implementation choices behind stable interfaces.

Agent Tools

Potential Agent tools include:

  • Knowledge retrieval.
  • File operations.
  • Git operations.
  • Documentation generation.
  • Email.
  • Calendar.
  • Infrastructure verification.
  • Service deployment.
  • Application-specific workflows.

Tool authority must remain explicit.

Near-Term Milestones

The immediate sequence is:

Milestone 1 — Gateway Base Commissioned

  • Host stable.
  • Docker verified.
  • Repository initialized.
  • .thoth documents committed.

Milestone 2 — Open WebUI Operational

  • Service deployed.
  • Data persistent.
  • Browser interface working.

Milestone 3 — Oracle Connected

  • Ollama reachable.
  • Models enumerated.
  • Gemma 4 available.

Milestone 4 — End-to-End Local Inference

  • Prompt submitted through Open WebUI.
  • Gemma 4 executes on Oracle.
  • Response returns to the browser.
  • Evidence recorded.

Milestone 5 — Operational Documentation

  • Service README complete.
  • Verification procedure complete.
  • Backup scope documented.
  • Recovery procedure documented.
  • Initial image version pinned.

Deferred Capabilities

The following capabilities are intentionally deferred until the relevant architectural foundations exist:

  • Public internet exposure.
  • Unrestricted external integrations.
  • Autonomous infrastructure modification.
  • Frontier-provider access to private archives.
  • Semantic retrieval without permission enforcement.
  • Centralized Agent execution without delegated authority.
  • Large-scale vector indexing.
  • Knowledge graph generation.
  • Multi-user organizational access.
  • High availability.
  • Complex orchestration frameworks.
  • Unbounded background automation.

Deferral protects the architecture from premature coupling.

Non-Goals

The roadmap does not aim to create:

  • A collection of every available AI tool.
  • A permanent dependency on Docker.
  • A permanent dependency on Open WebUI.
  • A permanent dependency on Ollama.
  • A permanent dependency on Gemma 4.
  • An unrestricted RAG system.
  • An autonomous Agent with broad administrative authority.
  • A model-owned knowledge store.
  • A hidden proprietary knowledge silo.
  • Infrastructure whose complexity exceeds its demonstrated need.
  • A system that treats installation as completion.

Milestone Philosophy

Each milestone should deliver independently valuable capability.

The system should remain useful if later phases are delayed or never completed.

Earlier phases should avoid unnecessary assumptions about later implementations.

A complete vertical workflow is preferred over many disconnected components.

Every milestone should produce:

  • A documented capability.
  • A verification path.
  • Operational evidence.
  • Recovery awareness.
  • Updated Project State.

Success Measures

Progress should be measured by:

  • Reproducibility.
  • Architectural stability.
  • End-to-end capability.
  • Knowledge independence.
  • Access correctness.
  • Egress control.
  • Provider replaceability.
  • Recovery readiness.
  • Operational legibility.
  • Security boundary clarity.
  • Ease of extension.
  • Long-term maintainability.

Container count is not a success measure.

Model count is not a success measure.

Feature count is not a success measure.

Roadmap Governance

This roadmap is expected to evolve.

Phases may be:

  • Added.
  • Reordered.
  • Split.
  • Combined.
  • Deferred.
  • Removed.

Changes should preserve the guiding sequence:

Establish
    ↓
Verify
    ↓
Preserve
    ↓
Retrieve
    ↓
Authorize
    ↓
Package
    ↓
Route
    ↓
Reason
    ↓
Act
    ↓
Audit

Reasoning and action occur only after the required knowledge and policy boundaries exist.

Final Vision

The Thoth Gateway infrastructure is not simply a Docker host.

It is the governed threshold between durable knowledge and the changing systems that use it.

Over time, Gateway should allow:

  • People to interact with preserved knowledge.
  • Local models to reason privately.
  • Frontier models to be used selectively.
  • Agents to perform constrained work.
  • Sources to remain traceable.
  • Permissions to remain enforceable.
  • Services to remain replaceable.
  • Infrastructure to remain recoverable.
  • Understanding to survive implementation changes.

Technology will continue to change.

Gateway should make those changes possible without sacrificing the knowledge, authority, and architectural clarity that Project Thoth exists to preserve.

Reference in New Issue View Git Blame Copy Permalink