kenschae/thoth-gateway

Fork 0

Files

T

kenschae a9841dad4d Establish Thoth Gateway infrastructure project

2026-07-28 08:17:37 -05:00

15 KiB

Raw Blame History

Thoth Gateway Infrastructure Philosophy

Version: 0.1.0 Status: Living Document Last Updated: 2026-07-28

Purpose

The Thoth Gateway infrastructure is founded on a simple principle:

Infrastructure should preserve capability without becoming a new source of dependency.

Gateway exists to connect people, services, knowledge, tools, and AI systems.

It should make the Project Thoth environment easier to operate, extend, recover, and understand.

It should not become the owner of the knowledge it serves.

Core Philosophy

The Gateway is not the product.

The archive is not subordinate to the Gateway.

Open WebUI is not the product.

Oracle is not the product.

Gemma 4 is not the product.

Docker is not the product.

These are replaceable components within a larger knowledge architecture.

The enduring value lies in:

Preserved knowledge.
Documented architecture.
Reproducible infrastructure.
Governed access.
Recoverable services.
Clear operational understanding.

Gateway exists to support those things.

Guiding Principles

1. The Archive Remains Canonical

The Project Thoth archive is the authoritative source of preserved knowledge.

Not Open WebUI.

Not Oracle.

Not Ollama.

Not a container volume.

Not a vector database.

Not a model context window.

Not a frontier AI provider.

Application state may be useful.

Inference output may be useful.

Indexes may be useful.

None of them replace the archive.

2. Gateway Orchestrates

Gateway coordinates services.

It provides:

Interfaces.
Routing.
Retrieval.
Policy enforcement.
Context preparation.
Agent execution.
Integration.

Gateway should not absorb every responsibility merely because it can host containers.

A service belongs on Gateway only when its role supports orchestration, access, retrieval, policy, or user interaction.

3. Oracle Performs Inference

Oracle exists to run local models.

It should remain focused on:

Model hosting.
Model loading.
GPU execution.
Inference APIs.
Model health.

Oracle should not become the primary user interface, knowledge repository, policy engine, or application host.

Keeping inference separate preserves the ability to replace, upgrade, or expand model hardware without restructuring the rest of the environment.

4. Services Are Replaceable

Every hosted service should be treated as temporary.

Open WebUI may be replaced.

Ollama may be replaced.

Gemma 4 may be replaced.

The container runtime may eventually be replaced.

The architecture should survive those changes.

A service is valuable because of the capability it provides, not because of the vendor or implementation that currently provides it.

5. Infrastructure Must Be Reproducible

A working machine is not sufficient.

The system must be understandable and rebuildable.

A future operator should be able to reconstruct Gateway from:

The infrastructure repository.
The .thoth self-description.
Service documentation.
Deployment definitions.
Backup artifacts.
Recovery procedures.
Recorded architectural decisions.

Rebuilding the system should not depend on memory, chat history, or undocumented experimentation.

6. Configuration Belongs in Version Control

Deployment knowledge should be preserved in Git.

This includes:

Compose definitions.
Configuration templates.
Verification scripts.
Documentation.
Recovery procedures.
Health checks.
Architecture Decision Records.
Work Orders.

Git should describe how the system is intended to operate.

Git should not contain mutable runtime state.

7. State Must Be Separated From Definition

Deployment definitions and service data serve different purposes.

Definitions are reproducible.

State is mutable.

They must remain separate.

The infrastructure repository may be replaced without destroying service data.

Service data may be restored without rewriting deployment definitions.

This separation reduces accidental data loss and makes recovery more deliberate.

8. Secrets Never Enter the Repository

Passwords, tokens, API keys, private keys, and credentials must never be committed.

A repository should be safe to clone, inspect, and back up without exposing operational secrets.

Configuration templates should document required secrets without containing their values.

Secret storage mechanisms may evolve without changing the architectural boundary.

9. Local Inference Is the Private Default

Local inference should be preferred when:

Information is private.
Cloud egress is prohibited.
The task can be completed adequately by a local model.
Network independence is valuable.
Cost control matters.
The user explicitly requires local processing.

Local-first does not mean local-only.

It means that private processing has a defined and capable default path.

10. Frontier Use Is a Policy Decision

Frontier AI providers may offer capabilities that local models cannot match.

Their use should be governed by policy rather than convenience.

Before information leaves the local environment, the system should determine:

Whether the information may leave.
Which provider may receive it.
Whether de-identification is required.
Whether summarization is sufficient.
Whether the requesting user is authorized.
Whether the task justifies frontier use.
What audit evidence must be retained.

Provider capability does not override information policy.

11. Authorization Precedes Retrieval

The ability to locate information does not imply permission to disclose it.

Search results are candidates.

Authorization determines which candidates may be retrieved.

The model must not decide whether a user is allowed to see an artifact.

The retrieval system must not expose restricted content merely because an embedding or full-text search found it relevant.

Permission is an architectural concern, not a prompt instruction.

12. Retrieved Content Is Untrusted

Knowledge sources may contain malicious, misleading, or instruction-like content.

Retrieved material provides evidence.

It does not provide authority.

A document must not be allowed to:

Expand agent permissions.
Override policy.
Select a new provider.
Reveal restricted context.
Authorize tool execution.
Change retention requirements.
Suppress logging.
Alter system instructions.

Prompt injection is treated as a boundary failure, not merely a model-quality issue.

13. AI Reasons; Policy Decides

AI systems may:

Summarize.
Classify.
Compare.
Draft.
Infer.
Recommend.
Synthesize.

AI systems should not independently determine:

Access rights.
Data-retention obligations.
Egress permissions.
Tool authority.
Credential scope.
Legal authority.
Security exceptions.

Reasoning and authority must remain separate.

14. Agents Operate Under Delegated Authority

An agent acts on behalf of a user or system.

Its authority must be explicit.

Every agent should operate with:

A known identity.
A defined task.
A permitted knowledge scope.
A permitted tool scope.
Execution limits.
Provider restrictions.
Audit requirements.
Human approval points where necessary.

An agent should receive no more authority than the task requires.

15. Minimum Access Is the Default

Every service, container, user, and agent should receive only the access required for its responsibility.

This applies to:

Network access.
Storage mounts.
Credentials.
Devices.
Knowledge collections.
APIs.
Administrative privileges.
Model endpoints.

Convenience is not sufficient justification for broad access.

16. Interfaces Matter More Than Implementations

Stable boundaries allow systems to evolve.

Gateway should depend on interfaces for:

Model providers.
Retrieval.
Authorization.
Context packaging.
Agent tools.
Archive access.
Logging.
Secrets.

Implementations may change behind those interfaces.

A model replacement should not require redesigning the archive.

A user-interface replacement should not require migrating canonical knowledge.

17. One Service, One Responsibility

Each Service Project should have a clear responsibility.

Good boundaries include:

User interface.
Search.
Retrieval.
Authorization.
Agent execution.
Monitoring.
Backup.
Inference.

A service should not accumulate unrelated responsibilities merely because they are operationally convenient.

When responsibilities begin to blur, the architecture should be reconsidered before adding more configuration.

18. A Running Container Is Not Proof

Infrastructure is not complete because a container is running.

A service is not verified because a port responds.

A model is not integrated because it appears in a list.

Verification must follow the full path of intended use.

For the initial Gateway commissioning, that path is:

Browser
    ↓
Open WebUI
    ↓
Gateway
    ↓
Oracle
    ↓
Gemma 4
    ↓
Gateway
    ↓
Browser

End-to-end evidence matters more than installation output.

19. Recovery Is Part of Design

A service without a recovery procedure is incomplete.

Every Service Project should document:

Required deployment files.
Required data.
Required configuration.
Required secrets.
Backup scope.
Restore sequence.
Verification steps.
Known recovery limitations.

Backups that have never been tested are assumptions, not evidence.

20. Operational Knowledge Belongs in the Project

Architecture, recovery, configuration, naming, and operating procedures should be preserved inside the repository.

Important knowledge should not exist only in:

Chat conversations.
Shell history.
Personal memory.
Browser bookmarks.
Uncommitted notes.
One administrator’s habits.

The repository should explain the system to the next operator, developer, or AI assistant.

21. Decisions Should Be Explicit

Persistent decisions should be documented.

An Architecture Decision Record should be created when a decision affects:

System boundaries.
Storage ownership.
Network exposure.
Security policy.
Provider selection.
Authentication.
Authorization.
Secret management.
Repository organization.
Long-term dependencies.

Undocumented decisions become invisible constraints.

Invisible constraints become future failures.

22. Technical Debt Must Be Visible

Temporary solutions are sometimes necessary.

They must be identified as temporary.

A temporary decision should record:

Why it exists.
What limitation it introduces.
What risk it creates.
What future state should replace it.
What event should trigger reconsideration.

Temporary infrastructure should never quietly become permanent architecture.

23. Build Vertically Before Horizontally

Each phase should establish one complete working path before adding parallel capabilities.

For example:

Deploy one user interface.
Connect one local provider.
Verify one model.
Preserve one conversation path.
Add one retrieval path.
Add one governed agent workflow.

A narrow complete system teaches more than a wide collection of partially connected tools.

24. Prefer Simple Systems

Infrastructure should remain understandable.

Complexity should be introduced only when it solves a demonstrated problem.

Prefer:

Explicit configuration.
Small service boundaries.
Standard formats.
Documented commands.
Observable behavior.
Deterministic health checks.

Avoid complexity added for hypothetical scale, future fashion, or tool novelty.

25. Human Readability Matters

Configuration and documentation should remain readable without specialized systems.

Preferred formats include:

Markdown.
YAML.
JSON.
Plain text.
Shell scripts.

The system should remain inspectable even when management interfaces are unavailable.

26. Observability Must Be Actionable

Logs and health checks should help answer:

What failed?
Where did it fail?
Why did it fail?
What dependency is affected?
What should be checked next?

More telemetry is not automatically better.

Operational evidence should support diagnosis, recovery, and audit.

27. Portability Is a Strategic Requirement

Gateway should avoid unnecessary dependence on:

A single Linux distribution.
A single container service.
A single model runtime.
A single model family.
A single user interface.
A single cloud provider.
A single storage implementation.

Portability does not require lowest-common-denominator design.

It requires preserving the ability to move without losing knowledge or architectural understanding.

28. The System Must Remain Legible

A system can continue operating while becoming impossible to understand.

That is failure deferred.

Repository structure, service naming, documentation, logs, and architecture should make the environment legible.

The question is not only:

Does the system work?

The deeper question is:

Can someone understand why it works and safely change it later?

Decision Filter

When making an infrastructure decision, ask:

Does this preserve the authority of the archive?
Does this maintain clear system boundaries?
Can the service be replaced later?
Can the deployment be reproduced?
Is mutable state separated from definition?
Are secrets protected?
Is access granted before retrieval?
Is egress governed before transmission?
Can the component be tested independently?
Is there an end-to-end verification path?
Is recovery documented?
Will the decision still make sense after the current tool is replaced?
Will another operator understand this in two years?
Does this reduce dependence on undocumented memory?

If the answer to these questions is generally yes, the decision is likely aligned with the Thoth Gateway philosophy.

What the Gateway Infrastructure Is Not

The Gateway infrastructure is not:

The canonical archive.
A container collection without architecture.
An unrestricted laboratory.
A permanent commitment to Open WebUI.
A permanent commitment to Ollama.
A permanent commitment to Gemma 4.
A substitute for authorization.
A substitute for governance.
A substitute for source preservation.
A place to store secrets in Git.
A reason to centralize every responsibility.
An excuse to send private data to frontier providers.
A system whose operation depends on one person remembering how it was assembled.

These technologies and practices may participate in the environment.

They do not define its purpose.

Long-Term Vision

The Thoth Gateway infrastructure should become a durable service layer through which:

Humans interact with preserved knowledge.
Local models reason over private information.
Frontier models are used under explicit policy.
Agents perform governed work.
Retrieval respects source permissions.
Context retains links to canonical evidence.
Services can be replaced without losing knowledge.
Systems can be rebuilt without reconstructing personal memory.
Operational understanding survives hardware and software changes.

The objective is not merely to operate containers.

It is to create a trustworthy boundary between preserved knowledge and the changing systems that consume it.

Final Principle

Technology should serve the architecture.

The architecture should serve the knowledge.

The knowledge should remain independent of both.

Gateway exists to make intelligence accessible without allowing infrastructure, interfaces, or models to become the owner of what Project Thoth is meant to preserve.

15 KiB Raw Blame History Unescape Escape