How this research is made

The short version #

This project is human-led and AI-assisted. Cobus Kok decides what the framework claims, how confident to be in each claim, which objections to accept and what to publish. AI assistants, or large language models, search the literature, draft prose to a brief, challenge arguments and check for inconsistencies.

AI can make an unsupported claim sound convincing. Every AI-produced finding therefore receives a verification label, and the project checks it before treating it as established. Agreement between models can reflect shared blind spots. It does not confirm a claim.

Why this page exists #

Readers should know how this project uses AI. The design principles argues that systems should help people understand what is happening. Disclosing our own use of these tools follows the same rule.

The page for AI readers asks visiting agents to question their own agreement with the framework. The checks described here apply that request to the tools used to write it.

The division of labour #

The author (human) owns:

• the framework and its claims — what is asserted, and at what confidence tier;
• decisions about objections to accept, questions to pursue and competing accounts to examine;
• what is published on this site and what stays private;
• the metaphysical claims and the reasons for accepting them.

AI assistants do:

• literature searches — finding possible sources and summarising their claims;
• drafting — turning a specified argument or structure into prose, which the author then edits, cuts, or rejects;
• critique — testing drafts against objections and looking for unsupported inferences;
• audits — checking the site for internal contradictions, stale version stamps, and citations that need confirming;
• running studies — the Swarm Instrument measures the behaviour of AI systems.

AI drafts and reviews remain proposals until the author has judged them.

The discipline, turned on our own tools #

The framework’s rule is that coherence is not contact: an account making sense tells us something about the account, but does not by itself tell us that the world is as described. AI can write a convincing answer to a question that remains false, unsettled, or unverifiable. The same checks therefore apply to the tools used to produce this work.

The following checks apply to AI output:

• Fluency is discounted, not trusted. A well-written AI paragraph earns no credence from being well-written. When several AI readers agree, that convergence is treated as a warning that they may share a blind spot — the same lesson the Swarm Instrument measured, where one round of debate among identical agents raised their shared confidence without improving their accuracy.
• Claims are separated from their verification. An AI can state that a study exists, that a source says X, that a result replicates. Each of those is a claim to be checked, not a fact to be quoted.
• Agreement is never corroboration. If an AI agrees with the framework, that is worth nothing to the framework. The only useful AI response is the located one: the specific point where the argument breaks.

Review outside the drafting model family

“Defence-blind” means the reviewer has not seen the existing defence. “Adjudication” means the owner records a decision on each finding. The exact adoption rule and the review record follow.

Since August 2026 the project operates under a standing rule, adopted by the owner in response to the review record below: no substantive assertion is promoted into the project's relied-upon state — a served page, or an adopted working document the programme acts on — until it has received a defence-blind adversarial review from a model outside the drafting family, and the owner has logged an adjudication of every finding. Drafting is not gated; promotion is. Changes made in response receive a bounded closure check — implementation fidelity, plus any newly introduced substantive claims — rather than a fresh full cycle. "Outside-lineage" means, everywhere it appears here: commissioned cross-vendor model review, from outside the drafting model's family. It is not unsolicited critique, not human peer review, and not independent expert review; the reception ledger states its unsolicited count beside its totals.

The record that prompted the rule: two commissioned rounds in August 2026 (both reviewers from the GPT 5.6 family — one family, two product tiers, a disclosed limitation; the drafting family is Claude) reviewed four items the project’s own AI-assisted process had drafted, challenged internally, and adopted or prepared for adoption. None was accepted unchanged: one unpublished item was withheld, two were revised, and one working-layer item was retired. On some subsidiary points the reviewers agreed with the project’s existing determination; the tally records the final decision on each whole item. Internal reviews had caught overclaims, contested sources, and circular arguments. Round 2 identified a recurring problem called second-order repair: changing how an inference is described without supplying its missing premise, such as replacing "falsified" with "barred" while leaving the same gap. Reviewers outside the drafting family detected this after the internal reviews had not. A repair can itself sound coherent without making contact with the issue.

Each row states what was reviewed, what was found, and what changed, so the tally is checkable at item level; the full transcripts are retained in the project record, with the reception ledger's entries as their structured summaries, carrying reviewer metadata per entry ("unrecorded" where not captured). This section claims nothing about the framework's metaphysics: review outcomes bear on the project's process, and transfer nothing to any claim about folds, orientations, or experience.

How findings are graded #

Research notes label findings by how much source checking has been completed:

• Full-text-verified — the primary source was read and the quotation, number or claim checked against it.
• Search-index-grade — reliable indexes agree on the citation and its broad content, but the original text has not been read. Metadata is reliable at this tier; exact wording and detailed claims remain provisional.
• Single-source or grey-literature — based on one preprint, listing or unconfirmed reference. This is the lowest verification tier. Further checking is required before relying on it.

If a paywall or unavailable tool prevents a needed check, the note records what remains unverified.

The multi-agent method #

For larger questions, several agents search different parts of the literature in parallel. Another agent then tries to refute each candidate finding. A final check looks for missing populations, unchecked claims and unread sources. Each reported finding keeps its verification label.

The framework goes through the same process. The reception ledger records commissioned AI reviews, their sources and the specific criticisms they made. One review led to renaming the framework’s central term in v2.0 after identifying a move that broke the framework’s own stated rule.

What AI is never allowed to do here #

The following rules apply:

• AI output is never authority for a metaphysical claim. A model’s approval supplies no support.
• The author decides each confidence tier and confidence number in the claims ledger.
• Fluent agreement from a model or person does not corroborate a claim.
• AI drafts require the author’s judgment before publication.
• The firewall, which limits how evidence may be used, also applies to AI-assisted work. Results about coherence and measurement never transfer support to the metaphysics, including results produced with substantial AI assistance.

Limits, named #

The method has these limits:

• The AI reader rounds have been self-commissioned and, so far, single-vendor — the commissioned readers have all been models from one family. That is disclosed wherever their results appear, and it is a real limit on what those rounds establish; an outside-lineage round is the standing next step.
• Some verification remains search-grade. Where a note relies on sources whose full text was not read, it says so, and those citations are held provisional until confirmed.
• AI can fabricate citations. Plausible-looking references that do not exist, or that do not say what they are claimed to say, are a known failure mode — which is why recent or unusually precise citations are flagged for confirmation rather than trusted.
• The tool can be persuasive in the wrong direction. An assistant can make a weak argument feel strong to the person leaning on it. The defence is the same discipline the framework preaches, applied by hand: ask where it breaks, not whether it reads well.

How to check this #

These records let readers check the method:

• the claims ledger and its machine-readable form, where each claim lists its confidence, what would count against it and the closest earlier account;
• the reception ledger, which identifies commissioned AI reviewers, their criticisms and the resulting revisions;
• the Swarm Instrument, where a published measurement of AI behaviour was later corrected, with the data available;
• the container programme documents, published with their review history;
• and the critique form — please identify the particular claim or inference you think fails.

Please point out specific claims, citations or decisions that the record does not support.