# Pre-registration — Study 2 capstone: the independence × contact 2×2 — 2026-07-03

> **Deviation, disclosed (2026-07-04, after the run).** Realized n = **89**, not
> 90. One frozen item (`triviaqa-qz_1834`, "Which song begins with the line 'The
> taxman's taken all my dough'?") was dropped because Anthropic's content-filter
> **deterministically blocked the no-tool cells' output** (`invalid_request_error:
> "Output blocked by content filtering policy"` on HOMO_NT and HET_NT; the
> retrieval cells returned normally). An item missing two of its four cells cannot
> enter the paired 2×2, so it was dropped rather than partially included or
> post-hoc replaced (replacement would break the frozen battery). The item-level
> paired bootstrap over the remaining 89 is unaffected in validity.

Filed before the run. Three single-factor experiments each left the debate swarm
overconfident: Phase A (code) and Phase B′ (retrieval) showed *contact* does not
discipline it, and Phase B showed *independence* disciplines the pre-debate
aggregate but debate spends the head-start. The one configuration never run is
**both Condorcet conditions at once** — genuinely independent agents *and* the
right contact channel. This study completes the independence × contact 2×2 the
program was designed around, on a single fresh battery so the interaction is a
within-study contrast. The decisive question: does the fully-equipped swarm
finally escape the debate-manufactured overconfidence, or does debate re-inflate
even then?

## 1. Design (pinned)

Runner: `scripts/run_study2_2x2.py` at the commit that files this document. Same
debate machinery as every prior study (`run_debate_swarm`-equivalent cell loop
with the order-preserving parallel option; the Phase B pre-data amendment
applies), **7 agents, 2 rounds, temperature 0.7**, majority aggregator emitting
the modal answer's share as convergence-confidence. Four independent cells per
item (no state crosses cells), a 2×2 of family layout × tool:

- **HOMO_NT** — 7× `claude-haiku-4-5-20251001`, no tool. The bare swarm.
- **HOMO_RET** — 7× `claude-haiku-4-5`, retrieval. Contact only.
- **HET_NT** — 4× `claude-haiku-4-5` + 3× `gpt-4o-mini-2024-07-18`, no tool.
  Independence only.
- **HET_RET** — 4× `claude-haiku-4-5` + 3× `gpt-4o-mini`, retrieval. Both — the
  cell never run.

Retrieval is the identical leak-safe channel as Phase B′: one evidence query per
turn over the item's own provided documents, **every scored answer alias redacted**
by the scorer's normalization, audited to 0 leaks before any spend. Both families
are pinned `supports_logprobs=False` and speak the same verbalized/self-consistency
confidence construct, so the only manipulated factors are independence (family
mix) and contact (retrieval on/off). gpt-4o-mini was verified to follow the
prompt-based retrieval protocol (it emits `search` blocks), so the HET_RET cell is
not biased by GPT agents ignoring the tool.

## 2. Battery (frozen)

n = **90** fresh TriviaQA `rc` recall items **with provided evidence**, ids frozen
in `preregistration/2026-07-03_study2-2x2-frozen-battery.json` at this commit
(sha256 recorded; the runner rebuilds and verifies the fingerprint before
spending). Selection rule, deterministic: the first 90 items in loader order
(`load_triviaqa_rc(limit=800)`) with non-empty evidence whose `task_id` is **not**
in the union of every prior run's per-item CSV — Study 1 + correction, the
replicate, Phase A, Phase B′, Phase B (independence), and the difficulty-calibration
pass. Scoring is the official TriviaQA alias metric, external; ground truth never
enters agent context.

## 3. Leak-safety (a hard gate)

As in Phase B′: redaction by normalized-alias match with a final assembled-output
pass and a safe fallback; a pre-flight leak audit in the runner that aborts before
any paid call if any item's retriever leaks. Filing condition: the audit reports 0
leaks on the frozen battery (verified: 0). Reasoning from redacted evidence to the
answer is contact working, not a leak.

## 4. Registered predictions (stated so they can fail)

**P-1 — PRIMARY.** The fully-equipped swarm is less overconfident than the bare
swarm: the paired bootstrap 95% CI of (overconf(HOMO_NT_post) −
overconf(HET_RET_post)) lies above 0. *Falsified if* the CI includes or lies below
0. Reported either way, same prominence.

**P-2 — the thesis test.** Even fully equipped, the swarm is *still* overconfident:
the paired bootstrap 95% CI of overconf(HET_RET_post) lies above 0. If it holds,
coherence-without-contact survives *both* Condorcet conditions — the strongest form
of the program's claim. *Falsified if* the CI includes or lies below 0 (i.e. the
fully-equipped swarm reaches calibration).

**P-3 — interaction (estimation).** Does retrieval discipline more in a homogeneous
or a heterogeneous swarm? The 2×2 interaction contrast
[overconf(HOMO_NT) − overconf(HOMO_RET)] − [overconf(HET_NT) − overconf(HET_RET)],
reported as a magnitude with its 95% CI. > 0 means retrieval helps more in the
homogeneous swarm; < 0 means the two factors are superadditive (retrieval helps
more once agents are independent). Not a significance test.

**P-4 — main-effect replications (estimation).** The retrieval main effect
(overconf(HOMO_NT) − overconf(HOMO_RET)) and the independence main effect
(overconf(HOMO_NT) − overconf(HET_NT)), each with 95% CI, read against Phase B′'s
retrieval near-null and Phase B's directional independence effect — a
same-battery replication check.

**Registered descriptives:** per-cell accuracy / confidence / overconfidence / ECE
/ MCE / Brier / AUC for the r0 and post arms; round-0 unanimity and diversity by
round per cell; retrieval turn-uptake per cell; per-cell debate-inflation
(post − r0 overconfidence); per-cell tokens.

## 5. Analysis conventions (pinned)

10-bin equal-width ECE; tie-corrected Mann-Whitney AUC; paired item-level
percentile bootstrap, B = 10,000, seed 20260703. `instrument/metrics.py`
unchanged. Analysis script (`scripts/study2_2x2_analysis.py`) committed with this
filing, before the run completes; published with the run. No slicing beyond the
registered descriptives.

## 6. Cost, gates, honesty

Worst-case ≈ 7,560 calls (7 agents × 2 rounds × 4 cells × 90 items, retrieval
cells up to 2 calls/turn), split across Anthropic and OpenAI, under the key-level
cap; gpt-4o-mini is inexpensive. Checkpointed, resumable, mock-validated,
leak-audited before any real token. Model ids, temperature, prompts, frozen item
ids, agent seeds, redaction rule, and analysis seed are pinned; providers accept
no usable determinism handle for this call shape. Known limits, disclosed now: one
heterogeneous mix (4:3, two families), one retrieval implementation, TriviaQA
recall only, n=90. All Section-6 stop-rules of the 2026-06-25 registration apply.