9.3 KiB
Thinking Methodology
Introduction
The Thinking Methodology Project is a way of using generative AI that developed from my own approach to AI conversations. Over time, I started to use generative AI as a collaborating agent. Where I can explore ideas, half-baked plans, comment on trends, or just vent about something I read online. Colloquially speaking, I would throw shit against the wall and see what sticks.
The more I used AI in this fashion, I noticed a couple of significant limitations to the technology and developed this project as a response. The first limitation is what I call Flattening and popular culture calls "AI Slop." The second was AI Independence. The need to be able to construct an inquiry in any engine and get similar results.
Flattening
If we begin with the assumption that Large Language Models are statistically based inference engines, then we may conclude that all inquiries posed to the inference engine will naturally move to the center of a distribution. This is a process that I refer to as Flattening.
During any given request to provide information, the inference engine will naturally reply with the most common (or likely) options. For example, if you ask your engine for the recipe for Chocolate Chip Cookies, you will most likely get the Tollhouse Recipe as it is the one that occurs most frequently in the training materials.
I began to see evidence of the flattening trend in the role-playing game space. There were recurring themes that were showing up in fictional settings over a short period of time. For example, if you ask GenAI to create a name for your fantasy setting you will likely get a name that includes something from this list:
- Ash / Ashes — Ash of the First Dawn, Kingdom of Ash, Ashen Crown, Ashes of Eternity. AI fantasy naming loves ash because it implies ruin, rebirth, fire, antiquity, and melancholy all at once.
- Shadow — Shadowborn, Crown of Shadows, Shadow of the Fallen. Probably the single easiest shorthand for “dark fantasy” without specifying what is actually dark.
- Blood — Blood Moon, Blood Crown, Bloodbound, Blood of the Ancients. Instantly supplies lineage, violence, sacrifice, and forbidden inheritance.
- The Fallen — fallen gods, fallen kingdoms, fallen stars, fallen kings, fallen empires. It conveniently creates a lost golden age without requiring much historical invention.
- The Last ___ — The Last Guardian, The Last Kingdom, The Last Mage, The Last Light. Creates immediate stakes and implied extinction.
- Forgotten / Lost — forgotten realms, forgotten gods, lost civilizations, lost heirs, lost temples. AI tends to generate worlds whose primary archaeological activity consists of rediscovering things everyone inexplicably forgot.
- Ancient awakening — an imprisoned god, primordial evil, sleeping dragon, buried civilization, or ancient machine is waking up. Something has generally been asleep for approximately 10,000 years and has chosen the player's tutorial level as the ideal moment to return.
- Corruption / Blight — magical corruption spreading through the land, forests dying, people mutating, black veins appearing in crystals. It provides a highly visible universal threat without requiring complicated politics.
- Chosen bearer of a mysterious power — the protagonist discovers an ancient mark, forbidden magic, unusual bloodline, mysterious artifact, or dormant ability that makes them uniquely important.
- Shattered world / broken cycle — shattered kingdoms, fractured moons, broken seals, sundered gods, disrupted cycles of death and rebirth. This gives the setting an immediate explanation for ruins, monsters, magic, and general apocalyptic scenery.
Like a Trope or a Stereotype, the LLM has learned from its training data that there are certain common categories in every field of inquiry. The result is that GenAI becomes recognizable in all forms because it is flattened - in popular culture this is being called "AI Slop".
AI Independence
In my career I have worked in a variety of regulated and confidential environments - finanacial services, banking, insurance, healthcare, and government. I work with both privacy data and intillectual property: data that is extremely sensitive and controlled.
In these environments, certain data simply cannot be submitted to an engine controlled by a third party where the organization bears significant risk of data exposure even if the third party provides privacy and security settings.
The logical conclusion is to stand up a local inference engine where sensitive information is exclusively processed. The complexity of architectural decisions for creating the local environment is out of the scope of this project.
The challenge, when utilizing more than one inference engine is that responses from each inference engine may vary wildly. In some circumstances this may be an advantage, but this project is concerned with consistency among engines. The idea is that I should be able to provide the same prompt to both a frontier and local AI and get similar (not exact) responses.
How Is This Different From Skills?
At this point, you may look at my introduction as be asking how this is different from skills offered on some frotier platforms. In short, these are not fundamentally different from skills. In fact, the implementation instructions take on the same execution model as skills.
The fundamental difference with this Thinking Methodology is that it combines a set of skills in a specific way to provide a complete end to end thinking process for the inference engine to follow. In this way, it is a skill orchestration method rather than the skills themselves.
Description of Components
The Thinking Methodology is implemented with a grouping of markdown instruction files. Each markdown file fulfills a particular component of the method. This section describes each of the 6 types of markdown documents that make up the methodology.
Constitutions
A constitution document is a markdown that provides the overarching instructions that the AI must follow for a particular line of reasoning. It defines HOW the AI must think.
Protocols
Protocols are refinements on the constitution that begin to address what role the AI is performing and HOW to engage in that role. Thus we have protocols for critical thinking, research assistant, learning companion, etc.
Voices
Voices define the narrative style and expression that the AI should follow when responding to inquiry. While it is partly about the "personality" that is displayed, the larger purpose is to make the AI sound less AI by providing details for breaking up common AI artifacts.
Artifacts
Artifact files define what you are asking the AI to produce. The conversation itself may be the product of the inquiry as is the case with ideation sessions. Other artifacts may be a long form blog article, a short social media post, or an interpretive essay.
Channels
Channels define where the output will ultimately be used. For example, the artifact may be a full blog article, but you may want to post that article to both a personal blog and substack. The channel files provide technical details about publishing expectations on each channel.
Perspectives
Perspectives may be the most important files in the methodology. These files directly address the issue of Flattening by providing a perspective by which the AI should operate. This is different from a role or protocol in that it tells the AI to view the query from a defined point of view.
Order of Execution
The idea behind the methodology is that the AI will consider each provided markdown in a specific order. Each layer builds on the last and when taken together provide a comprehensive thinking instruction that the AI can follow.
The constitution tells is the guiding principles, the protocols give it the roles, the voice tells it how to speak, artifacts and channels tell it what and where to produce output and the perspective shakes it up.
- Constitution
- Protocols
- Voice
- Artifact
- Channel
- Perspectives
- Additional Sources (CAG)
File Architecture
This Thinking Methodology is implemented as a system of markdown files. The files are organized into folders that align with the thinking components.
/artifacts
/channels
/constitutions
/protocols
/perspectives
contrarian.md
extraterrestrial.md
left-hand-of-darkness.md
goffman.md
raw.md
TAG mapping
One way to include the files in a prompt is to use metadata or even a section that lists the files for the AI to consider. However, the easiest method is to map each file to a tag. This is similar to how skills are implemented in many major platforms.
Thus my prompt may begin or end with a sequence of tags that identify the files to review. For example, if I am writing a blog article for my website the tag may look like
[COI][CI][WS][WS-BLOG][WS-WEB][RAW]
| Filename | Tag |
|---|---|
| constitution-of-inquiry.md | [COI] |
| critical-inquiry-protocol.md | [CI] |
| withered-sanctum-voice.md | [WS] |
| withered-sanctum-blog.md | [WS-BLOG] |
| withered-sanctum-website.md | [WS-WEB] |
| raw-perspective.md | [RAW] |
Alternatively, a comma-separated list may be an easier approach. The key is to make sure the AI knows how to map the tag to files that need to be reviewed.
[COI,CI,WS,WS-BLOG,WS-WEB,RAW]