kenschae/thoth-gateway

Fork 0

Files

T

kenschae a9841dad4d Establish Thoth Gateway infrastructure project

2026-07-28 08:17:37 -05:00

27 KiB

Raw Permalink Blame History

Thoth Gateway Infrastructure Architecture

Version: 0.1.0 Status: Living Document Last Updated: 2026-07-28

Purpose

This document describes the canonical architecture of the Thoth Gateway infrastructure.

It is intended to answer a single question:

How do users, services, knowledge, and AI inference flow through the Thoth Gateway environment?

This document defines architecture, responsibilities, boundaries, and information flow.

It does not describe current implementation status.

Current implementation status is maintained separately in:

.thoth/00-project-state.md

Implementation procedures belong in service documentation, scripts, Work Orders, and Architecture Decision Records.

Architectural Context

The Thoth Gateway infrastructure provides the service and orchestration layer for Project Thoth.

The environment separates three primary concerns:

Knowledge preservation and retrieval.
Service orchestration and user interaction.
AI inference.

These concerns must remain independently replaceable.

The Gateway does not own the knowledge archive.

Oracle does not own the knowledge archive.

Open WebUI does not own the knowledge archive.

AI providers consume approved context supplied by Project Thoth services.

Architectural Principles

The Thoth Gateway infrastructure is built upon the following principles:

Gateway orchestrates services.
Oracle performs local inference.
The archive remains canonical.
AI systems are consumers, not repositories.
Services are replaceable.
Persistent data is separated from deployment definitions.
Deployment definitions are version controlled.
Runtime state is not stored in Git.
Secrets are not stored in Git.
Local and frontier inference are accessed through explicit provider interfaces.
Information access is governed before retrieval.
Information egress is governed before inference.
Components should be loosely coupled.
Components should be independently testable.
Significant architectural decisions must be documented.
A rebuilt system should require project documentation, not personal memory.

High-Level Architecture

                           Human Users
                               │
                               ▼
                       Gateway Interfaces
                               │
                 ┌─────────────┴─────────────┐
                 ▼                           ▼
             Open WebUI                Future Clients
                 │
                 ▼
          Gateway Service Layer
                 │
      ┌──────────┼───────────┬───────────────┐
      ▼          ▼           ▼               ▼
 Retrieval   Agent Runtime   Policy       Integrations
 Services                    Layer
      │          │           │               │
      └──────────┴─────┬─────┴───────────────┘
                       ▼
                 Context Package
                       │
                 ┌─────┴─────┐
                 ▼           ▼
          Local Provider   Frontier Provider
             Interface        Interface
                 │                 │
                 ▼                 ▼
          Oracle / Ollama     Approved Cloud AI
                 │
                 ▼
              Gemma 4

Primary Systems

1. Thoth Gateway

thoth-gateway is the orchestration and service-interface host.

Gateway responsibilities include:

Hosting containerized services.
Providing user-facing interfaces.
Hosting retrieval services.
Hosting agent orchestration services.
Enforcing access and routing policy.
Preparing context for AI consumers.
Routing inference requests.
Maintaining service configuration.
Maintaining operational logs.
Coordinating integrations with Project Thoth repositories.

Gateway does not:

Serve as the canonical knowledge archive.
Own canonical source artifacts.
Perform primary GPU inference.
Determine authorization solely from model output.
Allow containers unrestricted access to all knowledge sources.
treat Open WebUI conversation history as the canonical archive.

2. Thoth Oracle

thoth-oracle is the local AI inference host.

Oracle responsibilities include:

Hosting the local model runtime.
Loading approved local models.
Performing GPU inference.
Exposing controlled model endpoints.
Reporting model availability.
Returning inference results to authorized Gateway services.

The initial Oracle runtime is Ollama.

The initial target model is:

gemma4:e4b

The initial endpoint is:

http://192.168.5.52:11434

Oracle does not:

Host the primary user interface.
Own the knowledge archive.
Determine whether a user may access a source artifact.
Search the archive independently unless explicitly provided that capability.
Route information to frontier AI providers.
Treat model memory or context as durable knowledge storage.

3. Project Thoth Knowledge Repository

The Project Thoth knowledge repository contains the durable knowledge assets used by the environment.

It may contain:

Canonical source artifacts.
Derived artifacts.
Metadata.
Manifests.
Harvests.
Relationships.
Search indexes.
Embeddings.
Access metadata.
Source locations.
Collection definitions.

The knowledge repository is independent of Open WebUI, Ollama, Gemma 4, and any frontier AI provider.

Services may consume knowledge from the repository only through approved access paths.

4. Open WebUI

Open WebUI is an initial user-facing AI interaction service hosted on Gateway.

Its responsibilities include:

Providing a browser-based conversation interface.
Presenting available models.
Sending inference requests to configured providers.
Displaying model responses.
Maintaining its own application state.
Supporting future integration with Gateway retrieval and agent services.

Open WebUI is a replaceable interface.

Open WebUI conversation data is application state, not automatically a canonical Project Thoth artifact.

Preservation of Open WebUI conversations requires a future Capture Connector or equivalent deterministic capture process.

Machine Responsibilities

Gateway Responsibilities

User interfaces
Container orchestration
Agent orchestration
Retrieval coordination
Policy enforcement
Context preparation
Inference routing
Service integrations
Operational monitoring

Oracle Responsibilities

Model runtime
Model loading
GPU inference
Local inference API
Model health reporting

Repository Responsibilities

Canonical source preservation
Derived artifact storage
Metadata organization
Relationship storage
Search preparation
Knowledge retrieval

Service Architecture

Each containerized application hosted by Gateway is treated as a Service Project.

A Service Project contains its deployment definition and documentation.

Typical structure:

service-name/
├── compose.yaml
├── .env.example
├── README.md
├── config/
└── scripts/

The canonical Compose filename is:

compose.yaml

A Service Project should define:

Service purpose.
Container image.
Network requirements.
Storage requirements.
Configuration requirements.
Secret requirements.
Dependencies.
Health verification.
Backup requirements.
Recovery procedure.
Upgrade procedure.
Known limitations.

Repository Architecture

The infrastructure repository is located at:

/srv/thoth/containers

Canonical structure:

/srv/thoth/
├── containers/
│   ├── .git/
│   ├── .thoth/
│   │   ├── 00-project-state.md
│   │   ├── 10-architecture.md
│   │   ├── 20-philosophy.md
│   │   ├── 30-glossary.md
│   │   ├── 40-coding-standards.md
│   │   └── 50-roadmap.md
│   ├── docs/
│   ├── scripts/
│   ├── open-webui/
│   │   ├── compose.yaml
│   │   ├── .env.example
│   │   └── README.md
│   ├── .gitignore
│   └── README.md
├── data/
├── config/
├── backups/
└── logs/

Repository Boundary

The Git repository contains reproducible infrastructure knowledge.

Git-Managed Artifacts

The following should normally be committed:

Compose definitions.
Configuration templates.
.env.example files.
Documentation.
Verification scripts.
Deployment scripts.
Backup scripts.
Recovery scripts.
Architecture Decision Records.
Work Orders.
Project self-description.
Health-check definitions.
Network diagrams.
Service dependency documentation.

Non-Git Runtime Artifacts

The following must not be committed:

Application databases.
User conversation data.
Container volumes.
Generated logs.
Backup archives.
Downloaded model files.
Temporary files.
Runtime caches.
Private keys.
Passwords.
Access tokens.
API keys.
Secret-bearing .env files.

Storage Architecture

Mutable runtime state is separated from deployment definitions.

/srv/thoth/containers
    Git-managed deployment definitions

/srv/thoth/data
    Persistent service application data

/srv/thoth/config
    Runtime configuration not appropriate for Git

/srv/thoth/backups
    Generated backup artifacts

/srv/thoth/logs
    Host-managed service and operational logs

Service data should be organized by Service Project.

Example:

/srv/thoth/data/open-webui

Open WebUI maps this host directory to:

/app/backend/data

The repository may be deleted and restored without deleting runtime data.

Runtime data may be restored without requiring changes to deployment definitions.

Configuration Architecture

Configuration is divided into three categories.

1. Public Configuration

Public configuration is safe to commit.

Examples:

Port assignments.
Container names.
Internal service names.
Non-sensitive endpoint templates.
Feature flags.
Health-check definitions.

Public configuration belongs in:

compose.yaml
Committed configuration files
.env.example

2. Environment-Specific Configuration

Environment-specific configuration varies by deployment but is not necessarily secret.

Examples:

Oracle IP address.
Gateway domain name.
Local storage paths.
Internal DNS names.
Deployment-specific port choices.

Environment-specific configuration may be stored in an uncommitted .env file.

A sanitized .env.example must document required variables.

3. Secrets

Secrets include:

Passwords.
API keys.
Private keys.
Authentication tokens.
Encryption keys.
Database credentials.
Frontier AI credentials.

Secrets must never be committed to Git.

The initial implementation may use locally protected environment files.

A future secrets-management service may replace local environment files without changing the Service Project interface.

Network Architecture

The initial local request path is:

User Browser
    │
    ▼
thoth-gateway
    │
    ▼
Open WebUI Container
    │
    ▼
192.168.5.52:11434
    │
    ▼
Ollama on thoth-oracle
    │
    ▼
gemma4:e4b

Initial known hosts:

Host	Address	Responsibility
`thoth-gateway`	`192.168.5.51`	Service orchestration and interfaces
`thoth-oracle`	`192.168.5.52`	Local AI inference
Local DNS	`192.168.5.25`	Internal name resolution

IP addresses represent the current environment and should eventually be replaced in service configuration by stable internal DNS names.

Container Network Architecture

Service Projects should use Docker-managed networks where practical.

Containers should not use host networking unless an explicit architectural reason is documented.

Network exposure should follow the principle of minimum required access.

A service should expose only ports required by:

Human users.
Other Gateway services.
Approved local systems.
Administrative operations.

Container-to-container communication should use service names on Docker networks where both services run on Gateway.

Communication with Oracle crosses the host boundary and currently uses the Oracle network address.

Information Flow

Direct Inference Flow

The initial direct inference path is:

1. A user submits a prompt through Open WebUI.
2. Open WebUI receives the prompt on Gateway.
3. Open WebUI sends the request to the configured Ollama endpoint.
4. Oracle loads or accesses the requested model.
5. Oracle performs inference.
6. Oracle returns the generated response.
7. Open WebUI presents the response to the user.
8. Open WebUI stores application state in its persistent data directory.

This flow does not yet include Project Thoth archive retrieval.

Future Knowledge-Augmented Flow

The intended knowledge-aware inference path is:

1. A user submits a request.
2. Gateway identifies the authenticated user.
3. The policy layer determines the user's permissions.
4. Retrieval services search only authorized repository content.
5. Source artifacts are resolved from their canonical locations.
6. The policy layer classifies the retrieved information.
7. The routing layer determines permitted inference providers.
8. A context package is assembled.
9. The approved AI provider receives the request and context.
10. The provider returns a response.
11. The response retains references to supporting source artifacts.
12. The interaction may be captured into the archive through a Capture Connector.

Authorization must occur before retrieval results are disclosed to the model or user.

Provider routing must occur before context leaves the local environment.

Retrieval Architecture

Project Thoth retrieval must separate four concerns:

Search
Authorization
Source Resolution
Context Packaging

Search

Search locates potentially relevant knowledge objects or artifact metadata.

Search results are candidates, not automatically authorized results.

Authorization

Authorization determines whether the requesting identity may access each candidate artifact.

Authorization is not delegated to the language model.

Source Resolution

Source resolution retrieves the canonical artifact from its current location.

The repository may store metadata and addresses without duplicating the original artifact.

Gateway must have authorized technical access to the artifact location.

Context Packaging

Context packaging selects and formats authorized information for a specific consumer.

The package may vary depending on:

User identity.
Model provider.
Data classification.
Token budget.
Task type.
Citation requirements.
Locality requirements.

Policy Architecture

The policy layer governs both access and egress.

Access Policy

Access policy determines:

Who may search a collection.
Who may retrieve an artifact.
Which portions of an artifact may be disclosed.
Which metadata may be exposed.
Whether inherited permissions remain valid.
Whether the requesting agent may act for the user.

Egress Policy

Egress policy determines:

Whether information must remain local.
Whether information may be sent to a frontier provider.
Whether information requires de-identification.
Whether information requires summarization before egress.
Whether a specific provider is approved.
Whether retrieval results contain restricted material.

Execution Policy

Execution policy determines:

Which tools an agent may invoke.
Which systems an agent may modify.
Whether human approval is required.
Which credentials may be used.
What audit evidence must be retained.

Provider Architecture

AI providers are accessed through provider interfaces.

Initial provider categories:

Local Provider
Frontier Provider
Specialized Provider

Local Provider

The initial local provider is Oracle through Ollama.

Local inference is preferred when:

Information is classified as private.
Information is prohibited from cloud egress.
Network independence is required.
A suitable local model is available.
Cost or latency favors local execution.

Frontier Provider

Frontier providers may be used when:

Policy allows information egress.
The provider is approved.
The task benefits from frontier capability.
Required context has been appropriately filtered.
Audit and privacy requirements are satisfied.

Provider Independence

Consumers should depend on a provider interface rather than hard-code model-specific behavior into the knowledge architecture.

A model replacement should not require restructuring the archive.

Agent Architecture

Agent services will be hosted or coordinated by Gateway.

An agent is treated as a governed consumer of tools and knowledge.

An agent request should contain:

Requesting identity.
Delegated authority.
Task objective.
Permitted tools.
Permitted knowledge scope.
Provider restrictions.
Execution limits.
Audit requirements.

Agents must not infer authorization from the existence of retrievable content.

Agents must not receive unrestricted credentials by default.

Agents must not treat model instructions as higher authority than system policy.

Security Boundaries

The environment contains several explicit trust boundaries.

Browser-to-Gateway Boundary

Gateway must not assume that browser input is trusted.

Relevant risks include:

Malicious prompts.
Uploaded files.
Cross-site requests.
Session theft.
Unauthorized users.
Prompt-injection content.

Gateway-to-Repository Boundary

Retrieval requests must be authorized.

The ability to locate an artifact does not grant permission to read it.

Gateway-to-Oracle Boundary

Only approved Gateway services should access Oracle inference endpoints.

Oracle should not be unnecessarily exposed beyond the local environment.

Gateway-to-Frontier Boundary

No context should cross this boundary until egress policy has approved it.

Container-to-Host Boundary

Containers should receive only the storage, network, device, and credential access required for their responsibility.

Prompt-Injection Boundary

Retrieved content is untrusted input.

A retrieved document may contain instructions intended to manipulate an AI consumer.

The retrieval system must distinguish between:

System policy.
User instructions.
Tool instructions.
Retrieved source content.
Generated content.

Retrieved artifacts provide evidence and context.

They do not provide execution authority.

A source artifact must not be permitted to:

Expand agent permissions.
Select a new model provider.
disclose restricted context.
Override system policy.
Authorize tool execution.
Change retention rules.
Modify audit requirements.

Persistence Architecture

Persistence is classified by role.

Canonical Persistence

Canonical Project Thoth artifacts are preserved in the knowledge repository.

Application Persistence

Application-specific state is stored under /srv/thoth/data.

Examples include:

Open WebUI users.
Open WebUI settings.
Open WebUI conversation history.
Service databases.
Service caches that must survive restart.

Application persistence is not automatically canonical.

Operational Persistence

Operational records may include:

Logs.
Health history.
Audit events.
Deployment records.
Backup records.

Operational persistence must have explicit retention rules.

Backup Architecture

Backups must preserve the ability to restore service operation without conflating deployment definitions with runtime state.

Repository Backup

The Git repository preserves:

Deployment definitions.
Documentation.
Scripts.
Architecture.
Standards.
Roadmap.
Project state.

Runtime Backup

Runtime backups preserve:

Service databases.
User state.
Configuration not stored in Git.
Required secrets through an approved secure mechanism.

Knowledge Backup

The knowledge repository follows its own preservation and integrity architecture.

Backing up Open WebUI does not back up Project Thoth.

Backing up Project Thoth does not automatically back up every application database.

Recovery Architecture

A complete recovery requires distinct restoration stages:

1. Restore or rebuild the Gateway operating system.
2. Restore the infrastructure repository.
3. Restore environment-specific configuration.
4. Restore required secrets.
5. Restore service runtime data.
6. Deploy Service Projects.
7. Validate network connectivity.
8. Validate Oracle access.
9. Validate service health.
10. Perform end-to-end workflow tests.

A Service Project is not considered recoverable unless its README documents its required data, configuration, secrets, and verification procedure.

Observability Architecture

Each Service Project should expose or document:

Container status.
Service health.
Dependency health.
Relevant logs.
Storage availability.
Endpoint reachability.
Backup status.
Version information.

The initial Gateway verification layers are:

Host Health
    ↓
Docker Health
    ↓
Container Health
    ↓
Service Health
    ↓
Dependency Health
    ↓
End-to-End Workflow Health

A running container does not prove that the hosted service works.

A responsive service does not prove that its dependency path works.

End-to-end verification is required for commissioning.

Testing Architecture

Testing should occur at multiple layers.

Host Tests

Examples:

Hostname verification.
Network configuration.
DNS resolution.
Route verification.
Storage mounts.
Reboot persistence.

Container Platform Tests

Examples:

Docker daemon status.
Non-root Docker access.
Compose validation.
Container restart behavior.
Network creation.
Volume access.

Service Tests

Examples:

Open WebUI HTTP response.
Application login.
Persistent state.
Configuration loading.
Health endpoint response.

Integration Tests

Examples:

Gateway reaches Oracle.
Gateway enumerates Oracle models.
Open WebUI lists Gemma 4.
Open WebUI submits an inference request.
Oracle returns a generated response.

End-to-End Tests

The initial end-to-end test is:

Browser → Open WebUI → Gateway → Oracle → Gemma 4 → Browser

Architectural Boundaries

Component	Primary Responsibility
Gateway Host	Run and coordinate infrastructure services
Docker Engine	Execute and isolate containers
Service Project	Define one deployable container application
Open WebUI	Provide a conversational user interface
Retrieval Service	Locate relevant knowledge
Policy Layer	Authorize access, routing, and execution
Context Packager	Prepare approved information for a consumer
Provider Interface	Abstract model-provider communication
Oracle	Execute local AI inference
Knowledge Repository	Preserve and organize knowledge
Capture Connector	Preserve external interactions as canonical artifacts
Agent Runtime	Coordinate governed multi-step work

No component should assume the primary responsibility of another.

Current Implementation Scope

The current implementation scope is intentionally narrow.

It includes:

Gateway base commissioning.
Docker Engine.
Docker Compose.
Infrastructure repository.
Project self-description.
Open WebUI.
Oracle connectivity.
Gemma 4 end-to-end inference.

It does not yet include:

Archive retrieval.
Permission-aware search.
Agent orchestration.
Frontier provider routing.
Automated Capture Connectors.
Centralized secrets management.
Full monitoring.
Automated backup scheduling.
Public ingress.
External identity integration.

Deferred capabilities must not be implied to exist merely because their architectural boundaries are documented.

Design Goals

The architecture is intended to achieve:

Reproducibility.
Service replaceability.
Provider independence.
Data durability.
Explicit authorization.
Controlled information egress.
Local-first private inference.
Independent testing.
Recoverability.
Maintainability.
Auditability.
Incremental evolution.
Long-term comprehensibility.

Non-Goals

The Gateway infrastructure is not intended to become:

The canonical knowledge archive.
An unrestricted container playground.
A single monolithic AI application.
A replacement for Project Thoth architecture.
A model-specific knowledge silo.
An implicit security boundary.
A repository for secrets.
A permanent dependency on Open WebUI.
A permanent dependency on Ollama.
A substitute for authorization policy.
A substitute for source preservation.

Architecture Decision Requirements

An Architecture Decision Record should be created when a decision:

Establishes a persistent system boundary.
Selects a long-lived platform dependency.
Changes repository structure.
Changes storage ownership.
Changes network exposure.
Changes authentication or authorization.
Changes provider-routing policy.
Changes secret-management strategy.
Changes canonical terminology.
Introduces a new infrastructure-wide standard.
Is likely to require explanation during a future rebuild.

Implementation details that do not establish a durable architectural constraint may remain in service documentation or Work Orders.

Relationship to Other Documents

This document defines how the Thoth Gateway infrastructure is organized.

Final Principle

Gateway exists to connect people, knowledge, tools, and intelligence without allowing any one implementation to become the owner of the system.

Services will change.

Models will change.

Interfaces will change.

The architecture must preserve the boundaries that allow each of them to be replaced without losing the knowledge, policy, and operational understanding that make the system trustworthy.

27 KiB Raw Permalink Blame History

Thoth Gateway Infrastructure Architecture

Purpose

Architectural Context

Architectural Principles

High-Level Architecture

Primary Systems

1. Thoth Gateway

2. Thoth Oracle

3. Project Thoth Knowledge Repository

4. Open WebUI

Machine Responsibilities

Gateway Responsibilities

Oracle Responsibilities

Repository Responsibilities

Service Architecture

Repository Architecture

Repository Boundary

Git-Managed Artifacts

Non-Git Runtime Artifacts

Storage Architecture

Configuration Architecture

1. Public Configuration

2. Environment-Specific Configuration

3. Secrets

Network Architecture

Container Network Architecture

Information Flow

Direct Inference Flow

Future Knowledge-Augmented Flow

Retrieval Architecture

Search

Authorization

Source Resolution

Context Packaging

Policy Architecture

Access Policy

Egress Policy

Execution Policy

Provider Architecture

Local Provider

Frontier Provider

Provider Independence

Agent Architecture

Security Boundaries

Browser-to-Gateway Boundary

Gateway-to-Repository Boundary

Gateway-to-Oracle Boundary

Gateway-to-Frontier Boundary

Container-to-Host Boundary

Prompt-Injection Boundary

Persistence Architecture

Canonical Persistence

Application Persistence

Operational Persistence

Backup Architecture

Repository Backup

Runtime Backup

Knowledge Backup

Recovery Architecture

Observability Architecture

Testing Architecture

Host Tests

Container Platform Tests

Service Tests

Integration Tests

End-to-End Tests

Architectural Boundaries

Current Implementation Scope

Design Goals

Non-Goals

Architecture Decision Requirements

Relationship to Other Documents

Final Principle

27 KiB

Raw Permalink Blame History