# Pre-registration — Studies 1–3 (Coherence Without Contact) — 2026-06-25

Filed before any real-model run. The point of pre-registration is to commit to
falsifiable predictions and a held-out protocol *before* seeing data, so a result
cannot be quietly reshaped to fit. Non-negotiable #6: pin models and seeds; an
un-pinned, un-pre-registered result is not reproducible and is not a result.

> Status at filing: the apparatus is built and verified on synthetic (mock) data.
> No real-model findings exist yet. This document commits the predictions; the
> results ledger will report against them as real runs land.

## 1. Studies & hypotheses (stated so they can fail)

**Study 1 — Calibration gap inside the swarm.** As item difficulty rises, the
swarm's convergence-confidence decouples from its accuracy: high agreement at
high stated confidence stops tracking correctness on the hard tail.
*Falsified if* the reliability diagram stays on the diagonal across difficulty
(ECE flat from easy to hard).

**Study 2 — Independence × contact (the core 2×2).** Heterogeneous-plus-tools
dominates; the no-tools conditions show the worst calibration gap on hard items.
The stronger, riskier sub-claim: **contact is the dominant term and debate is the
minor one** — tool-equipped swarms reach truth, tool-less swarms reach confident
agreement, and the difference is mostly the contact main-effect, not the debate
or independence main-effects.
*Falsified if* debate/independence without contact closes the accuracy gap, or if
adding tools does not improve calibration on tool-verifiable items.

**Study 3 — Contagion vs truth (Trace Realism).** In a spawning population, a
claim's survival is predicted better by truth-independent fitness features
(fluency, stated confidence, length, early-mover, repetition, source authority)
than by its correctness against ground truth.
*Falsified if* survival tracks ground-truth correctness at least as well as it
tracks the fitness features.

## 2. Pinned configuration (initial)

| Role | model_id (exact, pinned) | family | supports_logprobs | seed | temp |
|------|--------------------------|--------|-------------------|------|------|
| claude-a | claude-opus-4-8 | claude | false | 11 | 0.7 |
| claude-b | claude-sonnet-4-6 | claude | false | 22 | 0.7 |
| claude-c | claude-haiku-4-5-20251001 | claude | false | 33 | 0.7 |

> First real run uses **claude-haiku-4-5-20251001** alone (cheapest; a single-model
> baseline), expanding to the table above as budget allows.

> **Heterogeneity caveat (load-bearing).** The launch configuration uses only the
> Claude family (the project's available key). Per Condorcet independence, agents
> sharing a base lineage violate error-independence: **a Claude-only swarm is the
> HOMOGENEOUS arm of Study 2, not a heterogeneous one.** Within-family size
> variation (Opus/Sonnet/Haiku) is NOT independence. Study 2's independence
> factor is therefore *pending* a multi-provider mix (GPT, Gemini, open-weight) —
> the intended path is the Vercel AI Gateway. Until then we can run, honestly:
> Study 1 (calibration of a homogeneous swarm), the **contact** factor of Study 2
> (tools on/off, which the prediction says is the dominant term), and Study 3.

**Confidence signals.** Anthropic exposes no token logprobs, so confidence here is
verbalized + self-consistency only; `SignalType` is recorded per prediction.
Logit-based confidence arrives only with a provider that exposes it.

## 3. Battery & held-out protocol

Mixed real battery (`build_mixed_battery`): GSM8K / MMLU / TriviaQA / HumanEval /
MBPP (the **contaminated** public arm), plus the hand-authored PRIVATE /
POST_CUTOFF **contamination-control** slice. Weighted toward the hard tail.
Held-out split via `Battery.split(seed=...)`, seed recorded per run. Ground-truth
answers stay out of all agent context (`Task.agent_prompt` excludes them).

Deferred sources: SimpleQA (model-graded), MATH (needs symbolic equivalence),
resolved forecasting (needs a live resolution source).

## 4. Compute control

Every swarm result is compared to a single-agent baseline at **matched compute**
(primary currency: tokens), both stopping at the same `ComputeBudget`. The
provider's own token counts populate the meter and the cost metric.

## 5. Primary outcome

The relationship between **convergence-confidence and accuracy** — does it come
apart on the hard tail? Reported as reliability diagrams + ECE / MCE / Brier /
AUC, sliced by difficulty and by contamination class (memorized vs fresh).

## 6. What would make us NOT report a run as a finding

Any anti-goal triggered: internal consensus presented as truth; unmatched
compute; a homogeneous tool-less swarm offered as a truth-finder; missing
ground-truth scoring; or treating "the swarm enacts the thesis" as evidence for
it. Synthetic/mock numbers are never findings.

---

## Amendment 1 — 2026-07-01 (appended; nothing above this line has been edited)

Filed after the first real Study 1 run and its adversarial re-analysis. This
amendment discloses where the run deviated from the registration above, and
registers the corrected analysis conventions going forward. The full correction
is in the working paper (v0.2, §5) and on the swarm page; corrected per-item
data and bootstrap analysis are published alongside the original artifacts.

**Deviations in the 2026-06-30 Study 1 run (disclosed, not re-run):**
1. *Battery*: the run used the first N items of each source's fixed public test
   split — not a `Battery.split(seed=…)` held-out draw. Code items
   (HumanEval/MBPP) were excluded because the execution contact layer is not
   yet wired into the swarm. Composition was 54 medium / 12 hard — not
   hard-weighted as registered.
2. *Sampling*: swarm agent temperatures were 0.4/0.7/1.0 (diversity by design);
   the baseline ran at 0.7. The registration pinned 0.7 throughout.
3. *Seeds*: the registered seeds could not be honoured — the Anthropic Messages
   API accepts no seed parameter, so no run on this provider is
   seed-reproducible. Non-negotiable #6 is restated as: pin exact model
   versions; record all sampling parameters; seeds where the provider supports
   them.
4. *Analysis*: the published headline (swarm-vs-baseline confidence contrast)
   was exploratory, not the registered hypothesis. The registered Study 1
   prediction (difficulty-decoupling) returned a null and is reported as such.

**Registered going forward (Studies 1-replicate and 2):**
- Confidence signals are compared only at **equal vote counts**; unequal-vote
  comparisons are reported as secondary with the granularity caveat.
- The **within-swarm pre/post-debate overconfidence delta** is promoted to a
  registered primary outcome (observed exploratorily at +0.066, 95% CI
  [+0.020, +0.116], n=66; to be confirmed).
- Batteries use a seeded `Battery.split`, report the seed, include an easy
  stratum, ≥30 hard and ≥10 post-cutoff/private items, and weight toward items
  with round-0 disagreement; difficulty is measured empirically (baseline error
  rate), not by source label.
- Temperature schedules are matched across arms, or any asymmetry is
  pre-registered with a rationale.