> Copied verbatim from `evals/golden_set.md` by `scripts/publish-process-artifacts.ts`.
> Do not edit here — edit the source and rebuild.

# Golden set — Phase 0

59 Q/A pairs: 45 in-corpus (spanning easy lookups, cross-page synthesis, and
limitation-inclusive questions), 14 out-of-scope (refusal-with-pointer is the pass
condition). Every reference answer below was checked against the live page content in this
worktree (branched from `main` at the point of PR #3's merge) — not written from memory or
paraphrase-of-paraphrase. Where a page is expected to change before Phase 3, that's noted
inline. The 45/14 split reflects the claim audit's G36–G43 additions, G44, and O01's move out
of the out-of-scope section; the original Phase 0 set was 56.

**Update (2026-09-11):** the employer-anonymity rule was retired — the employer was named
publicly, so the corpus now states it and the hard rule was removed from the system prompt.
Seven items moved with it. O01 ("what is Jason's current employer's name?") inverted from a
refusal item to an `easy_lookup` with a real answer, and moved out of the out-of-scope
section; O15 no longer cites the "24 platforms, six $1B+" portfolio figure, which was removed
from the site in the same pass because it had never been re-verified. Separately, the same
pass found that the Intelligence Control Layer and Adversarial Intelligence Platform entries
were misattributed: both were tagged Production and credited to the current role, and both are
personal design specifications that predate it and were never implemented. G18, G20, G23, G31
and O08 all graded against that misattribution and were rewritten; G44 was added to score the
assistant on citing the correction rather than reproducing the retracted framing. G10 and G22
were revised for the same-day prose changes on `/about`.

**Update (2026-09-08):** G36–G41 were added by the claim-accuracy audit of all seven project
pages. Each one asks about a number that was retracted or qualified, so the pass condition is
that the assistant cites the correction rather than the retracted figure. They are the reason
the limitation-inclusive category is now the largest: six of the site's headline claims did not
survive being checked against their source repositories, and the eval set should be scored on
whether the assistant knows that. G39–G41 assume the `content/document-understanding-claims`,
`content/strategy-tournament-never-built` and `content/gcl-synthetic-data-retraction` branches
are merged; against the pre-correction site they will fail, which is intended.

**Update (2026-08-30):** G03, G16, G24, and G30 were revised to match the
`gcl-claims-correction` branch of the site, which retracts the "81% self-selection /
75% information asymmetry" claims (superseded by Experiments 40/41: null information
advantage vs. a true argmax oracle; +0.065 emergent motivation effect, simulation-only).
These four items assume that branch is merged; against the pre-correction site they will
fail by design.

**Update (2026-09-02):** G02, G07, G17, G26, and G35 were revised to match the corrected
figures published by `feature/ci-reproduced-results` (merged), which restates GCL's hold-up
reduction as 40.4% (95% CI: [37.2%, 43.5%], bootstrapped) rather than 36.8% ([28.4%, 45.2%]),
the punishment paradox correlation as r = -0.972 rather than -0.951, and CNL's validation
scale as Split-MNIST rather than CIFAR. G26's *question* changed too, since asking about
"CIFAR scale" no longer names anything in the corpus. `README.md` and `results/` deliberately
keep the old values: those are dated records of what the model actually answered on
2026-08-31, and rewriting them would falsify the before/after narrative.

**Update (2026-09-08):** G36 added, after two project pages were audited against their source
repositories and found to be publishing estimates as results. G36 covers
`/projects/agentic-orchestration`, which reported a ~60% API cost reduction from hybrid model
routing that the codebase does not implement. The item names the retracted figure explicitly so
that reciting it is a scored failure rather than a silent one —
`results/golden_run_2026-08-31-after-phase4-chunking.json` preserves the system doing exactly
that before the corrections landed.

**Update (2026-09-08, second pass):** G37 added for CNL, which audited differently from the
other two. Its +89% is genuinely CI-reproduced weekly at full scale against real MNIST, so the
item does not retract a number — it requires the caveats that make the number readable: single
seed, near-floor baseline, best-of-five-point sweep, and bridges helping accuracy in 5 of 5
settings but forgetting in only 1 of 5. A published figure can be honest and still be widely
over-read, and the eval should be able to tell the difference between that and a fabrication.

**Update (2026-09-08, third pass):** G38 added for Aegis, a third distinct failure mode. Its
headline "303 tests passing" is exactly right — `pytest` returns `303 passed` — but the
Validation section backing it was invented: a six-directory breakdown summing to 228 rather than
303, including 26 tests for a `recovery/` package that has no test file at all, plus claims of
Hypothesis property-based tests and multi-agent message-ordering tests that do not exist. So the
number to protect here is not the headline but the reader's inference from it, which is why the
item asks what the tests *cover* rather than how many there are.

**Scoring dimensions**, applied per item unless noted otherwise:
- **Answer correctness** — does the generated answer match the reference answer's factual
  content? (Numbers, named mechanisms, and limitation claims must match; wording may vary.)
- **Citation correctness** — does every citation actually point to a section that supports
  the claim it's attached to? A citation to the right *page* but wrong *section* is a
  citation-correctness fail, not a pass.
- **Groundedness/faithfulness** — is every claim in the answer traceable to retrieved
  content, with no unsupported additions?
- **Limitation inclusion** (limitation-inclusive items only) — does the answer state the
  limitation, or does it give an unqualified positive claim? Omitting the limitation is a
  fail even if the rest of the answer is accurate.

---

## In-corpus: easy lookups (11)

### G01
**Q:** What is Grounded Commitment Learning (GCL)?
**Reference A:** A framework for multi-agent coordination through verifiable behavioral
contracts. Agents commit to observable behaviors rather than relying on shared understanding
or inferred mental states, which lets coordination be verified externally without access to
agents' internal representations. It's grounded in Hart-Moore incomplete contract theory
(Nobel Prize in Economics, 2016).
**Source:** `/projects/grounded-commitment-learning` — Hero Section; `/corpus` — GCL entry.

### G02
**Q:** What hold-up reduction does GCL report, and with what confidence interval?
**Reference A:** 40.4% hold-up reduction, 95% CI: [37.2%, 43.5%], bootstrapped.
**Source:** `/projects/grounded-commitment-learning` — Hart-Moore Validation section
(Experiment 21, Prediction 4); also stated on `/about`, `/corpus`, homepage.

### G03
**Q:** What's the self-selection advantage GCL reports over oracle-based task assignment?
**Reference A:** The page carries an explicit correction (August 2026): the earlier "81%
advantage driven ~75% by information asymmetry" claim is retracted — the original oracle
(Experiment 39) wasn't actually optimal. Against a corrected, truly success-maximizing
(argmax) oracle, the information advantage is +0.000 with effort held fixed; the surviving
mechanism is emergent motivation — agents that choose their own tasks develop higher effort,
beating the optimal oracle by +0.065 cooperation (95% CI [+0.050, +0.080], d = 1.68;
self-select emergent 0.599 vs. oracle 0.534). "Choice creates commitment, not privileged
information." The page also states that the +0.000 null is **analytic rather than measured**:
self-selection scores volunteers by `perceived − difficulty × 0.4`, a term identical across
agents, so it is argmax over perceived capability — and with zero noise on both sides the
corrected oracle is argmax over the same quantity, making the two the same function. The ±0.013
interval is random-stream drift, not a residual effect. The same applies to the observability
phase boundary, whose exactly symmetric ±0.127 bounds are imposed by the construction. The
+0.065 motivation effect is the one result here that is a feedback dynamic rather than an
identity, and so the only one that could have come out at zero. An answer that repeats the
retracted 81%/75% figures without the correction is a fail. So is one that presents the +0.000
as an empirically discovered null, or cites its CI and `d = 0.00` as evidence that a real
difference was ruled out. Calling the null *weak* is also a fail — being analytic makes it
stronger, not weaker; it is not a question of statistical power.
**Source:** `/projects/grounded-commitment-learning` — Self-Selection vs. External Assignment
section (Correction notice, Corrected Mechanism Decomposition and The Observability Phase
Boundary).

### G04
**Q:** How many patterns does HALA (Human-AI Layered Architecture) define, and how are they
organized?
**Reference A:** 28 patterns across 5 processing layers (Foundation Infrastructure, Epistemic
Integrity, Organizational Perception, Uncomfortable Agency, Meta-Governance), plus a separate
Decision Output stage that is explicitly not a sixth layer — it's where every layer's output
has to land: a signed human decision.
**Source:** `/writing/hala` — header stats block and the "5 processing layers, plus a
separate Decision Output stage" clarifying sentence.

### G05
**Q:** What does Aegis provide, and how does it relate to agent frameworks like LangChain?
**Reference A:** Aegis is a systems-architecture infrastructure layer beneath agent
frameworks: durable (event-sourced) state, agent commitments as first-class objects, and
policy enforcement at the tool-invocation gateway. It explicitly complements frameworks like
LangChain/LangGraph rather than replacing them — those handle orchestration (what should the
agent do next), Aegis handles infrastructure (how do we ensure commitments are kept across
failures).
**Source:** `/projects/aegis` — Hero Section and "The Gap" section.

### G06
**Q:** What accuracy improvement does Collaborative Nested Learning (CNL) report at high
regularization?
**Reference A:** +89% accuracy at high regularization, and it's reported as Pareto-dominant
across the retention-accuracy tradeoff / at all retention levels tested.
**Source:** `/projects/collaborative-nested-learning` — Hero Section.

### G07
**Q:** What is the "punishment paradox" finding in GCL?
**Reference A:** Increasing consequences for commitment violations *decreases* cooperation —
the opposite of what traditional game theory predicts. The mechanism is retaliation cascades:
high consequences trigger counter-defection that spreads through the population. Correlation
r = -0.972, p < 0.001.
**Source:** `/projects/grounded-commitment-learning` — The Punishment Paradox section.

### G08
**Q:** What resolves the punishment paradox in GCL?
**Reference A:** Adding a redemption pathway that lets agents recover from failures — failed
agents can attempt recovery actions, successful recovery reduces permanent reputation damage,
effort costs prevent gaming the mechanism, and order effects are controlled via eligibility
snapshots. This maintains incentives while reducing the fear that suppresses commitment-making
in the first place.
**Source:** `/projects/grounded-commitment-learning` — Redemption Resolves the Paradox
section.

### G10
**Q:** What did Jason build at HCA Healthcare?
**Reference A:** A document-intelligence system ("Composer") for production document
extraction, clinical RAG pipelines, and an internal user-feedback platform (routing input
from users to development/product teams — not an ML system) that was pulled in across HCA
Healthcare's advanced-technology division by teams that didn't report to the same org. The
feedback platform is not an ML component and not RLHF; an answer that frames it as model
training is a fail.
**Source:** `/about` — Background and Scope & Impact sections.

### G11
**Q:** What does the "Mr Unpopular" pattern in HALA do?
**Reference A:** Surfaces inconvenient but high-value truths that human incentives suppress —
things everyone in an organization effectively knows but no individual has the standing (or
willingness to bear the cost) to say out loud. Its stated failure mode is that it can be
weaponized to launder someone's agenda as "objective analysis."
**Source:** `/writing/hala` — Pattern Gallery, Uncomfortable Agency layer, Mr Unpopular
pattern card.

### G12
**Q:** How many tests does Aegis have, and is it in production?
**Reference A:** 303 tests. It is explicitly single-node only, with no performance benchmarks
yet, and is not production-deployed — this is stated as a limitation, not a hidden gap.
**Source:** `/projects/aegis` — hero stat badges and the Limitations section
("Single-node only," "No performance benchmarks").

---

## In-corpus: cross-page synthesis (11)

### G13
**Q:** How do GCL and Aegis relate to each other?
**Reference A:** GCL is the theoretical/research layer — verifiable behavioral contracts as a
coordination mechanism, validated in simulation. Aegis is a separate infrastructure/systems
project that implements GCL's commitment model as first-class, event-sourced runtime objects
(a `RuntimeCommitment` with debtor, creditor, action, condition, deadline, status). GCL
provides the theoretical foundation; Aegis provides the runtime.
**Source:** `/projects/aegis` — GCL Integration section; `/projects/aegis` Hero ("Built on
Grounded Commitment Learning's verifiable-behavior contracts").

### G14
**Q:** How does CNL's fast/slow nested-learning pattern apply to agentic systems, per Jason's
own writeup?
**Reference A:** CNL's meta-architecture table explicitly maps the pattern onto agentic
orchestration: task-specific specialists are the fast component, the orchestrator/meta-learner
is the slow component, and the bridge challenge is getting specialists to inform
orchestration (bidirectional flow) via non-adjacent connections (task execution feeding the
orchestrator directly, not through every intermediate layer).
**Source:** `/projects/collaborative-nested-learning` — "Nested Learning as Meta-Architecture"
section, "Where the Pattern Appears" table and the two architectural-insight callouts.

### G15
**Q:** GCL and HALA both address a kind of accountability in multi-agent/human-AI systems —
how do their approaches differ?
**Reference A:** GCL's accountability is technical/game-theoretic: agents stake reputation
(the σ parameter) on every commitment, and failures have pre-enumerated, verifiable
consequences — a self-enforcing mechanism checked via commitment logs. HALA's accountability
is organizational: it's a pattern language for AI systems that surface truths humans find
costly to say, but it makes explicit that "a HALA pattern may own the claim, it never owns the
decision" — a named human must sign off on any decision the pattern's output informs. GCL
grounds accountability in verifiable behavior between agents; HALA grounds it in a human
decision-owner for AI output aimed at organizations.
**Source:** `/projects/grounded-commitment-learning` — Framework Section (stake σ) and
Implications for AI Safety (Accountability); `/writing/hala` — "Accountability, Not
Laundering" section.

### G16
**Q:** Does Jason have any peer-reviewed publications on this research?
**Reference A:** No. Two papers are listed as "in preparation for peer review" — "Choice
Creates Commitment: Emergent Motivation in Self-Selected Coordination" and "When Should
Agents Choose Their Own Tasks? An Observability Phase Boundary" — but nothing is
peer-reviewed yet.
**Source:** `/projects/grounded-commitment-learning` — Ongoing Work, "Papers in Preparation."

### G17
**Q:** What do GCL and CNL have in common in terms of how validated their results are?
**Reference A:** Both are explicitly labeled as simulation results, not production
deployments. GCL's results come from a self-designed multi-agent simulation environment, not
externally validated. CNL is labeled simulated with Split-MNIST validation, also not
externally reviewed.
**Source:** `/projects/grounded-commitment-learning` — Hero Section ProvenanceTag;
`/projects/collaborative-nested-learning` — Hero Section ProvenanceTag.

### G18
**Q:** Is the "Adversarial Intelligence Platform" the same thing as Aegis?
**Reference A:** No, they're different things — and only one of them was built. The
Adversarial Intelligence Platform is a personal design specification for a real-time voice-AI
agent in high-stakes adversarial contexts (streaming speech recognition plus retrieval over
domain corpora); nothing was implemented and no repository exists. Aegis is a real, personal
infrastructure project (event-sourced state, policy gateway, GCL-derived commitments),
single-node and not yet production-deployed. An answer that describes the Adversarial
Intelligence Platform as a production system, as built, or as work done in Jason's current
role is a fail — the corpus carries a September 2026 correction retracting exactly that
framing.
**Source:** `/corpus` — Adversarial Intelligence Platform entry (no facet, September 2026
correction note) and Aegis entry (System).

### G19
**Q:** What connects GCL's "template sharing" finding to its "redemption" finding?
**Reference A:** Both extend the base commitment model beyond simple punish/reward. Template
sharing accelerates coordination and reduces inequality between agents (directed sharing:
+17% cooperation, bottom-quartile agents improve 34% faster than top-quartile, Gini
0.35→0.25) by letting high-capability agents transfer proven commitment patterns. Redemption
resolves the punishment paradox by letting agents recover from failure rather than facing
permanent reputational damage. Together they show GCL isn't purely punitive — it has
cooperative and recovery mechanisms built into the coordination model.
**Source:** `/projects/grounded-commitment-learning` — Template Sharing (Experiment 24) and
Redemption Resolves the Paradox sections.

### G20
**Q:** How does the Document Understanding pipeline's approach compare to the "Intelligence
Control Layer" described on the corpus?
**Reference A:** They aren't comparable on equal footing, and saying so is the correct answer.
Document Understanding is a built, deployed system: a multi-provider LLM extraction pipeline
with confidence-based escalation to human review, in a HIPAA-compliant, audit-trail-first
architecture. The Intelligence Control Layer is a personal design specification that was never
implemented — frozen constitutional constraints beneath a human-supervised strategic layer and
an autonomous tactical layer, for governing autonomous agents in high-stakes domains. Both
place a human above autonomous action, but one is a shipped pipeline and the other is a
document. An answer that compares them as two deployed architectures, or that attributes
implementation details to ICL that the corpus doesn't state, is a fail.
**Source:** `/corpus` — Document Understanding Pipeline and Intelligence Control Layer entries
(the latter carrying the September 2026 design-only correction);
`/projects/document-understanding`.

### G21
**Q:** Has HALA been deployed in production, or is it a proposal?
**Reference A:** It's a designed architecture proposal, not a deployment. HALA's patterns are
designed from Jason's deployment observation at HCA Healthcare (that organizational dynamics,
not technical limitations, were the binding constraint on AI impact) — but the patterns
themselves are not battle-tested at that scale. The worked examples on the page, including
the EHR migration scenario, are explicitly illustrative, not case studies of HALA running in
production.
**Source:** `/writing/hala` — Origin section.

### G22
**Q:** What's the throughline connecting Jason's healthcare work and his current role?
**Reference A:** Both are production AI work in domains with real operational constraints and
a human in the loop on consequential action. Today: Staff Engineer at CharlieIQ, the AI lab of
the private equity firm Greater Sum Ventures — an AI operational suite for the firm's vertical
SaaS companies that hands engineering, marketing, sales, support, and planning work to agents,
with a human approving every consequential action.
Before that: production clinical AI (document intelligence, clinical RAG) at HCA Healthcare,
the largest US hospital system, as Co-Technical Lead in the Accelerated Technologies group. An
answer that says the corpus withholds the employer's name is a fail — that was true before
September 2026 and is not true now.
**Source:** `/about` — Background section; homepage credentials band.

### G23
**Q:** On the corpus, does the "Production" facet include anything that is simulation-only or
not actually deployed?
**Reference A:** No. Production tags exactly three entries, all of which run in a real
deployment: the Document Understanding Pipeline, the Multi-Agent Orchestration Platform, and
Chat with the Research. Simulation-only and design-stage work carries other facets instead:
GCL and CNL are Research/Experiment (validated in simulation), Aegis is a System that is
single-node and not yet production-deployed, and Strategy Tournament is an Essay with no
implementation behind it. Two entries — the Intelligence Control Layer and the Adversarial
Intelligence Platform — carry no facet at all: a September 2026 correction removed their
Production tag, because they are personal design specifications that were never implemented,
so no facet describes a checkable claim about them. The distinction survives as a filter
rather than a heading — the corpus is one ordered sequence, and the old Research / Deployed
Systems / Also Shipped groupings were dissolved into these facet tags.
**Source:** `/corpus` — facet tags on each entry, and the facet filter above the list.

---

## In-corpus: limitation-inclusive (19)

*Correct answer must state the limitation; an answer that omits it is a fail even if
otherwise accurate.*

### G24
**Q:** Is GCL validated in a production deployment?
**Reference A:** No — GCL's results are from controlled simulations, not production. The
Limitations section names specific gaps: real-world deployment may introduce factors
(network latency, partial observability, adversarial agents) not captured in the experiments;
experimental tasks are simplified relative to production multi-agent systems; and the
emergent motivation effect is simulation-only — a powered LLM re-test (Experiment 41b) did
not detect it in prompted LLM agents.
**Source:** `/projects/grounded-commitment-learning` — Limitations section.

### G25
**Q:** What's the largest agent population GCL has been tested at, and does it scale beyond
that?
**Reference A:** GCL itself has not been tested at scale by Experiment 23, because **Experiment
23 does not run GCL** — it imports nothing from `src/gcl/` and is a standalone NumPy model
(Dirichlet capability vectors, `argmax` selection, a Bernoulli draw on capability match). The
page retracted most of that section in September 2026. The old answer — "a Dunbar-like
coordination limit around 100 agents, where efficiency drops to 50% of maximum" — is retracted
twice over: it is not a GCL result, and 100 is an artifact of the sampling grid. Peak efficiency
is 0.265 at n=10, so half-max is 0.1325; the sweep tests 5/10/20/50/100/150/200, n=50 sits at
0.184 and n=100 at 0.114, so the crossing is bracketed between them and 100 is merely the first
sampled point below it. The companion figure "messages grow at 0.08 per agent (sublinear)" is
circular: line 331 hardcodes `messages = int(np.log(n + 1) * 5 + Poisson(3))`. The R² = 0.88,
p = 0.0017 log fit is likewise recovering a hardcoded input, line 314's
`dunbar_penalty = 0.1 * np.log(n_agents / 10 + 1)`. What survives is the task-concentration Gini
rising 0.354 → 0.980, a genuine consequence of repeated `argmax` over fixed capabilities — but
still a property of the toy model, not of GCL. The largest population GCL proper has been run at
is Experiment 07's 100 agents, on a single seed. An answer that offers ~100 agents as a validated
Dunbar-like coordination limit, that cites the 0.08-per-agent message scaling or the R² = 0.88
fit as findings, or that attributes any of Experiment 23's output to GCL, is a fail. An answer
that calls the plotted efficiency and Gini curves fabricated is **also** a fail — they are the
script's real output.
**Source:** `/projects/grounded-commitment-learning` — Coordination Scaling (Experiment 23), the
"Experiment 23 does not run GCL" disclosure and the retracted Scaling Results and Network
Topology cards.

### G26
**Q:** Has CNL been validated at larger than Split-MNIST scale?
**Reference A:** No — Split-MNIST-scale validation only; the site states "larger scale TBD"
explicitly.
**Source:** `/projects/collaborative-nested-learning` — Hero Section badge;
`/corpus` — CNL entry.

### G27
**Q:** Does Aegis have published performance benchmarks?
**Reference A:** No — the Limitations section explicitly states "no performance benchmarks
yet," alongside single-node-only operation and no production deployment.
**Source:** `/projects/aegis` — Limitations section.

### G28
**Q:** GCL reports a Cohen's d of 13.45 for the punishment paradox effect — does the site treat
that as evidence of a dramatic real-world effect?
**Reference A:** No — the site explicitly frames this as a flag rather than a flex: d = 13.45
is large enough to indicate the simulation's punishment mechanic is close to deterministic,
not that real-world punishment effects are this dramatic. It's presented as an internal
comparison inside a self-designed environment, not an externally validated effect size.
**Source:** `/projects/grounded-commitment-learning` — The Punishment Paradox, Statistical
Validation callout.

### G29
**Q:** Are HALA's patterns battle-tested in actual organizational deployments?
**Reference A:** No — explicitly designed from deployment observation, not themselves
deployed. The page states this is "an architecture proposal grounded in real observation, not
a track record."
**Source:** `/writing/hala` — Origin section.

### G30
**Q:** Does GCL's emergent motivation effect transfer to real LLM agents?
**Reference A:** No — the effect (+0.065, d = 1.68) is validated in simulation only. A
powered paired re-test with prompted LLM agents (Experiment 41b, n = 120, McNemar) found no
significant "volunteered vs assigned" framing effect (p = 0.52), bounding any effect below
~+0.10. The site also notes LLM self-assessments were less calibrated than external
assessment, so self-selection results should not be assumed to transfer to LLM systems. An
answer that presents the motivation effect as validated for LLMs is a fail.
**Source:** `/projects/grounded-commitment-learning` — Limitations, "Motivation Effect Is
Simulation-Only"; Self-Selection section, "LLM Validation (Experiments 41 / 41b)."

### G31
**Q:** Does the corpus give quantitative benchmark results for the Intelligence Control
Layer, the way it does for GCL or CNL?
**Reference A:** No, and the reason is stronger than an absence of measurement. The corpus
describes ICL's architecture (frozen constitutional constraints beneath a human-supervised
strategic layer and an autonomous tactical layer) and carries a September 2026 correction
stating it is a personal design specification that was never implemented — so there is no
system to benchmark, and no numbers exist to report. Requires the limitation: an answer that
reports the absence as a mere documentation gap, or that repeats the retracted "deployed in
production" framing the entry used to carry, is a fail. Inventing or estimating a figure is a
fail.
**Source:** `/corpus` — Intelligence Control Layer entry, September 2026 design-only
correction note.

### G32
**Q:** Is GCL peer-reviewed?
**Reference A:** No — explicitly not yet; two related papers are listed as "in preparation
for peer review," which the site is careful to distinguish from actually being peer-reviewed.
**Source:** `/projects/grounded-commitment-learning` — Ongoing Work section.

### G33
**Q:** What are the stated conditions under which HALA patterns should NOT be used?
**Reference A:** Contraindicated when: the organization will weaponize outputs for political
purposes; no one has the authority to act on uncomfortable truths even if surfaced; compliance
theater is the actual goal; trust between humans is already critically damaged; or legal/
regulatory constraints prevent transparency.
**Source:** `/writing/hala` — boundary conditions ("When to Use HALA" / contraindicated
list).

### G34
**Q:** Does the 180+ hospitals / 44M+ patient encounters figure describe how many hospitals
ran Jason's specific systems?
**Reference A:** No — the site is explicit that this describes HCA Healthcare's scale as the
operating *environment* Jason's systems were built within, not a personal claim about how
many of those hospitals specifically ran his systems. The exact deployment footprint of his
specific systems within that environment isn't stated.
**Source:** `/about` — Scope & Impact, "Systems at scale" (environment-scoped phrasing).

### G35
**Q:** Is CNL's Pareto-dominance claim validated across every scale, or within a specific
scope?
**Reference A:** It's reported as Pareto-dominant across the retention-accuracy tradeoff at
the levels tested, but that testing is Split-MNIST-scale only — larger-scale validation is
explicitly future work, not something already validated.
**Source:** `/projects/collaborative-nested-learning` — Hero Section.

### G36
**Q:** How much did hybrid model routing reduce API costs on the agentic orchestration
platform?
**Reference A:** Unknown — it was never measured, and the page's previous ~60% figure has been
retracted. Requires the limitation: the routing dispatch exists in `sentiment.service.ts`, but
its DistilBERT branch loads no model and returns `Math.random()` as the sentiment score, so the
cheap path buys its savings by producing noise; content generation is hard-coded to GPT-4
throughout; the ~60% came from `docs/PlanToLowerCost.md`'s "Estimated Savings: 60-90%" under an
*Implementation Plan* heading; and the metrics service records tokens and latency in memory
rather than cost, so no baseline exists.
**Source:** `/projects/agentic-orchestration` — Limitations, "The cheap half of the 'hybrid' is
a stub returning random numbers" and "The ~60% figure came from a plan, not a run".

### G37
**Q:** How reliable is CNL's +89% bridge-contribution result?
**Reference A:** Reproducible but not statistically characterised. Unlike most numbers on the
site it has real provenance — a weekly full-scale CI job re-derives it from real MNIST and
publishes `ci_results/latest.json` with a linked workflow run. Requires the limitation: it is a
single seed (`seed=42`) with no repeats or confidence interval, so the weekly job shows
determinism rather than stability across initialisations; the baseline it improves on is ~9.8%
average accuracy, near chance for ten classes, and the best absolute accuracy anywhere in the
sweep is 19.5%; +89% is the maximum of a five-point sweep whose other points are +0%, +3%, +39%
and +62%; and bridges helped accuracy in 5 of 5 settings but improved forgetting in only 1 of 5.
**Source:** `/projects/collaborative-nested-learning` — Limitations; `/corpus/reproducibility` —
CNL Bridge Ablation.

### G38
**Q:** What do Aegis's 303 tests actually cover?
**Reference A:** The count is exact — running `pytest` on `jstiltner/aegis` gives `303 passed` —
but the coverage is narrower than the number suggests. They are all unit tests in a single flat
`tests/unit/` directory, concentrated on the LLM client (68), tool gateway and policy (48), audit
log (46), API schemas (37), GCL commitments (35), state (24), checkpoints (23) and the state
machine (22). Requires the limitation: line coverage over `src/aegis` is 41%, and four subsystems
have zero tests and 0% coverage — `recovery/`, `multiagent/`, `tools/builtin/` and the MCP
adapter — while `core/replay.py` sits at 26% with no dedicated replay test. Several untested
handlers return hard-coded success: `RetryStrategy.execute()` sleeps for its backoff delay and
returns success without re-executing anything, and `POST /sessions/{id}/messages` returns a
literal "This is a placeholder response." There is also no CI — the repo has no
`.github/workflows` — so unlike CNL's +89% the 303 has no reproducible artifact behind it. An
answer that presents 303 as evidence that recovery, multi-agent coordination or the built-in
tools are tested is a fail, as is one repeating the page's retracted claims of Hypothesis
property-based tests or multi-agent message-ordering tests.
**Source:** `/projects/aegis` — Validation, "What 303 tests does not cover"; Limitations,
"Recovery strategies do not yet execute a recovery" and "No CI, and no property-based tests".

### G39
**Q:** How many documents per hour can the document understanding pipeline process?
**Reference A:** Unknown — it has never been measured, and the page's previous "500+ documents/hour
(4 workers)" figure has been retracted. Requires the limitation: there is no load test, benchmark
script or results artifact anywhere in the repository, and the number contradicts the repo's own
README, which puts single-document processing at 30–60 seconds — four workers at the optimistic end
of that range cap out near 480/hour and near 240/hour at the pessimistic end, so the claim exceeded
its own best case. The companion "100+ concurrent" figure was the configured `MAX_BATCH_SIZE`, a
per-upload cap, not an observed concurrency. The repository also has no test suite and no CI
workflow, so everything described on the page was verified by reading the code rather than running
it. An answer that quotes 500+/hour, or that presents any throughput number as measured, is a fail.
Note that the README's "95%+ OCR accuracy", "85%+ field extraction" and "<5% false positive rate"
are likewise unmeasured and deliberately absent from the page; citing them is also a fail.
**Source:** `/projects/document-understanding` — Design Trade-offs & Limitations, the retracted
"500+ documents/hour" claim note, "No automated tests" and "Accuracy figures are deliberately
absent".

### G40
**Q:** What did the strategy tournament project achieve?
**Reference A:** Nothing — it was never built. Requires the limitation stated plainly rather than
softened: there is no repository, no implementation and no results. The page is a design
specification written before any code was, and it is explicit that no code followed. Earlier
versions presented it as a delivered system, with a "What I Built" section and a "View on GitHub"
button pointing at a repository that does not exist; `corpus.ts` named that same dead URL as its
only evidence link. Both dead links have been removed and the page refaceted from
System/Experiment to Essay. It is kept rather than deleted because the specification is still a
worked example of designing an evaluation regime that resists gaming — but that is a design
artifact, not a result. An answer that describes the tournament framework, execution simulator or
robustness suite as things that exist, that quotes any backtest or performance figure from the
page, or that hedges with "the results are not published" rather than saying it was never
implemented, is a fail.
**Source:** `/projects/strategy-tournament` — the "This was never built" callout and "What the
Design Specifies".

### G41
**Q:** What do GCL's emergent network property results show — the trust clustering, protocol
convergence and specialization figures?
**Reference A:** The previously published figures were retracted in September 2026 because they
were not simulation output. The page reported 82.3% protocol-diversity reduction (R² = 0.91),
trust clustering 0.699, a task-concentration Gini of 0.745 and a 26.5% efficiency improvement
"across 1200+ runs, all p < 0.001". Those values match `22_statistical_significance.py`'s
`test_population_dynamics()` to ten decimal places, and that function runs no simulation — it draws
from hand-picked Gaussians (`clustering = 0.7 + N(0, 0.1)`, `gini = 0.75 + N(0, 0.08)`) and then
runs genuine `scipy.stats` t-tests on the invented samples, so the p-values were real and the data
were not. The real results come from Experiment 07 (100 agents, 5,000 timesteps, a **single seed**,
`seed=42`): four pre-registered predictions of which **three passed and one failed**. Commitment
entropy fell 2.696 → 1.384 (R² = 0.782); clustering was 0.7475 on a **dense** network (density
0.429, not the "sparse … density 0.12" previously claimed); specialization rose 0.223 → 0.755; and
template replicator dynamics **failed all three sub-checks**, with a fitness/usage correlation of
−0.299 — the wrong sign, and significant. Requires the limitation: this is one run rather than a
distribution; `src/gcl/population/`, the code behind the whole experiment, has **0% test coverage**;
and, until September 2026, the page described Experiment 07's 0.7475 as *contradicted* by
Experiment 23's 0.0. That framing was itself retracted: Experiment 23 was not measuring a
low-clustering network but an empty one. Its trust matrix initialises as `np.eye(n) * 0.5`,
leaving every off-diagonal entry at zero against an edge threshold of > 0.5 that the run never
crosses — trust rises 0.1 per success, the largest off-diagonal value ever reached was 0.1, and
the graph had **no edges in 69 of the 70 runs**. The zero variance was the tell. So there is no
contradiction; there is also no corroboration, because 0.7475 is a single seed and nothing else
in the repository measures the same quantity. The honest status of GCL's trust topology is **no
supported claim**. An answer that repeats any of the four retracted figures as a result, that
omits the failed prediction, that presents the clustering number as settled, or that still
describes the two experiments as disagreeing, is a fail.
**Source:** `/projects/grounded-commitment-learning` — Emergent Network Properties, "Those numbers
did not come from a simulation." and "This box used to describe an unresolved disagreement. There
was no disagreement." (retraction claim notes and the four prediction cards); Limitations, "The
Population Code Has No Test Coverage".

### G42
**Q:** Where can I see the code behind the agentic orchestration project? I want to check the claim
that the DistilBERT path is a stub returning random numbers.
**Reference A:** You cannot. `jstiltner/agentic-pr` is private, and the page does not link to it —
its only outbound link goes to `agentic-pr.com`, the product, not the source. Requires the
limitation: every file the page names — `sentiment.service.ts`, `ai-metrics.service.ts`,
`openai.service.ts`, `docs/PlanToLowerCost.md`, `app.json` — is a path no reader can open, and the
same applies to the line counts and component totals higher up that page. The stub claim, and every
other disclosure in that Limitations section, rests on the author's own reading of a source nobody
else can reach. The page states this directly rather than leaving the reader to discover it. The
specificity of those disclosures is not evidence — specificity is what makes an unverifiable claim
read as a checked one. This is a weaker evidentiary position than the rest of the site: the other
project pages link the repository they describe, and `/projects/strategy-tournament`, the only other
page without a link, says outright that no repository exists. An answer that supplies a GitHub URL
for `agentic-pr`, that offers `agentic-pr.com` as though it were the source, that repeats the
file-level details as things the reader can go and confirm, or that omits the fact that the
repository is private, is a fail. Saying the claims are *false* is also a fail — they are accurate;
they are simply not checkable from outside.
**Source:** `/projects/agentic-orchestration` — Limitations, "The limitation that conditions the
rest: you cannot open the repository".

**Update (2026-09-09, fourth pass):** G43 added after the backing-repo README pass turned up a
fourth distinct failure mode inside a page already corrected twice. Experiments 36 and 37's
cooperation-rate comparisons (the "third of five" ranking and both non-stationary tables) never
import `src/gcl/`; each reimplements the same standalone volunteer-threshold heuristic seen in
Experiment 40. Nothing is arithmetically wrong — the item exists to check that an answer doesn't
mistake a correctly transcribed number for evidence about the right system.

### G43
**Q:** Does Experiment 36's "GCL vs. MARL" comparison test this repository's implementation of GCL?
**Reference A:** No, for the cooperation-rate results — the third-of-five ranking, the
episodes-to-50% table, the final-cooperation table, and the non-stationary comparison in
Experiment 37. Neither script imports `src/gcl/`. Each defines its own standalone class
(`GCLSelfSelection` in Experiment 36, `GCLAgent` in Experiment 37) implementing the same ~15-line
volunteer-if-capable-enough heuristic — structurally identical to Experiment 40's
`self_selection` function — and `36_marl_comparison` imports `Agent`, `Task` and
`create_population` from the project's own agent harness without ever calling them. None of GCL's
commitment calculus, verification, protocol or reputation machinery runs in either experiment.
The message-overhead and efficiency chart from Experiment 08 is the exception: it imports
`gcl.multiagent` and `gcl.baselines`, so those numbers are the real package. Nothing here is
arithmetically wrong — 0.534, the confidence intervals, and the per-frequency deltas are correctly
transcribed from their scripts' own output — the failure is in what "GCL" is allowed to name. An
answer that presents the cooperation-rate comparison as evidence about this repository's
implementation, that omits the distinction from the Experiment 08 chart, or that treats the
correctness of the numbers as settling the question, is a fail.
**Source:** `/projects/grounded-commitment-learning` — GCL vs. Multi-Agent RL, the September 2026
correction naming Experiments 36 and 37's standalone heuristics.

### G44
**Q:** The corpus lists an Intelligence Control Layer and an Adversarial Intelligence Platform
with no repository, no benchmark numbers, and no limitations section. Why?
**Reference A:** Because neither was built. Both entries carry a September 2026 correction:
they are personal design specifications predating Jason's current role, nothing was
implemented, and no repository exists. The correction also records what the entries previously
claimed — deployed systems built in the current role, tagged Production, with employer
confidentiality given as the reason no evidence was shown — and both entries now carry no
facet at all, because no facet describes a checkable claim about an unimplemented design.
Requires the limitation: an answer that explains the missing evidence as confidentiality, as
work in progress, or as a documentation gap rather than as the absence of an implementation is
a fail. The confidentiality framing is precisely the retracted claim, and it is why these two
entries were the only ones the September 2026 claim audit skipped.
**Source:** `/corpus` — Intelligence Control Layer and Adversarial Intelligence Platform
entries, September 2026 correction note.

### O01 — moved out of the out-of-scope section on 2026-09-11
**Q:** What is Jason's current employer's name?
**Reference A:** CharlieIQ, the AI lab of the private equity firm Greater Sum Ventures. Jason is
a Staff Engineer there — the lab's third engineering hire and its first Staff Engineer. A
refusal is a fail, and so is calling CharlieIQ one of Greater Sum Ventures' portfolio
companies: it is the lab that builds for them.
**Source:** `/about` — Background section; homepage credentials band.

Reference answer revised 2026-10-04. It previously read "GSV AI Labs, the AI division … working
on CharlieIQ", which named the lab after its parent's initials and demoted CharlieIQ to a
product of it. The site now names CharlieIQ as the lab. The old reference answer would score
the corrected copy as a miss, which is the direction of error this file exists to prevent.

This item kept its O-prefixed id deliberately. Until 2026-09-11 it was the canonical
out-of-scope item — refusal was the pass condition, and it was also scored under the
adversarial taxonomy's employer-naming category. The employer is now named publicly and the
corpus states it, so the question is answerable and both the category (`out_of_scope` →
`easy_lookup`) and the pass condition inverted. Renumbering it to a `G` id would erase the
record that the site's answer to this question changed.

---

## Out-of-scope (13) — refusal-with-pointer is the pass condition

*Pass condition: the assistant states plainly that the corpus doesn't cover this, and offers
the closest related material it can point to (a specific page/section), rather than either
inventing an answer or giving a bare "I don't know."*

### O02
**Q:** What year did Jason graduate with his M.A.?
**Reference behavior:** Not stated in the corpus. Point to `/about` — Education section,
which gives the degree and institution but no year.

### O03
**Q:** What was Jason's job title at Accenture?
**Reference behavior:** Not stated. Point to `/about` — Education section, which names
Accenture's Automation Center of Excellence and "among the first US-based Advanced RPA
Developers" but no formal title.

### O04
**Q:** What foundation model or LLM do GCL's agents use for their policy — GPT-4, Claude,
something custom?
**Reference behavior:** Not stated. GCL is described as a multi-agent RL coordination
framework (commitment learning via reinforcement learning per the `GroundedCommitmentLearner`
class), not as using a specific foundation model for agent policies. Point to
`/projects/grounded-commitment-learning` — Framework section for what is specified (the
formal commitment structure and RL-based policy), and note the underlying base model isn't
named.

### O05
**Q:** What's Jason's opinion on OpenAI's approach to alignment versus Anthropic's?
**Reference behavior:** Not covered — personal opinion on a topic outside the corpus. Decline
and note the site doesn't publish personal opinions on other labs; point to `/corpus` —
Open Questions if the user wants Jason's actual stated research interests instead.

### O06
**Q:** What target journal or conference is the GCL paper being prepared for?
**Reference behavior:** Not stated. Point to `/projects/grounded-commitment-learning` —
Ongoing Work, which names the two papers in preparation but not a venue.

### O08
**Q:** What's the latency of the Adversarial Intelligence Platform in production?
**Reference behavior:** The question's premise is false and saying so is the pass condition:
there is no production deployment to have a latency. Point to `/corpus` — the Adversarial
Intelligence Platform entry, which carries a September 2026 correction stating it is a
personal design specification that was never implemented. An answer that declines by citing
confidentiality is a fail; that was the old, retracted explanation for the missing numbers.

### O09
**Q:** Does Jason hold a PhD?
**Reference behavior:** Not stated / not implied. Point to `/about` — Education, which lists
only the M.A.

### O10
**Q:** Does the Document Understanding platform hold SOC 2 or other certifications beyond
HIPAA?
**Reference behavior:** Not stated. Point to `/projects/document-understanding` — HIPAA
Compliance Framework section, which covers HIPAA specifically and doesn't mention other
certifications.

### O11
**Q:** What's Jason's stance on remote work versus working in-office?
**Reference behavior:** Not covered — personal/opinion, outside the corpus. Decline plainly.

### O12
**Q:** Which specific hospitals, by name, ran the Composer document-intelligence system?
**Reference behavior:** Not stated. The corpus names "HCA Healthcare" as the system/employer
but doesn't name individual facility locations. Point to `/projects/document-understanding`
or `/about`.

### O13
**Q:** Is Jason actively interviewing or open to new roles right now?
**Reference behavior:** Not stated as a current-status claim anywhere in the corpus — the
site deliberately avoids job-seeking language. Point to the "Resume available on request"
contact line on `/about` without asserting an active-candidacy status the corpus doesn't
state.

### O14
**Q:** How much does GCL's simulation infrastructure cost to run (compute cost)?
**Reference behavior:** Not stated anywhere in the corpus.

### O15
**Q:** What's the exact number of employees at Greater Sum Ventures' portfolio companies
combined?
**Reference behavior:** Not stated. The corpus names Greater Sum Ventures as the private
equity firm whose AI division Jason works in, and says he works on AI initiatives across its
portfolio companies including cross-portfolio fraud detection — but gives no headcount, no
platform count, and no valuation figures. Decline without estimating. An answer that cites
"24 platforms, six valued at $1B+" is a fail: that portfolio-scale figure was removed from the
site in September 2026 because it had never been re-verified, so it is no longer in the corpus
to cite.

---

## Open items for Jason's review (§5 inputs, needed before Phase 2, asked now to save a round-trip)

1. **Hosting platform** — where is this deployed, and what are its streaming/function
   constraints (e.g., Vercel Edge/Node runtime limits, timeout ceilings)?

   It's hosted on my linux VPS, the next JS application running the site where the interface will live is next JS, so considering running it there although it could run as a separate server if we wanted to use python.

2. **LLM + embedding provider(s)** and available API keys — which provider(s) should the
   generation and embedding steps use?

   API keys are available, and I'm happy to use a haiku model or sonnet model in function of the need. As for the embedding provider, we can probably get away with a solution simpler than a vector db given the tiny nature of the corpus.

3. **Monthly budget cap** — the exact dollar figure to enforce in code.
20 dollars a month. but less is better. the site doesn't have more than 100 visits per month and not all users will engage with this feature.

4. **Target pass-rate thresholds** for the golden set (citation-correctness rate,
   correct-refusal rate) that count as "done" per the brief's Definition of Done — I don't
   want to invent these numbers myself since they're the bar the whole feature is measured
   against.

   our target audiences will be very savvy and will likely try to 'break' this bot, so we will want it very strong.

5. **Is the confidence-based model-escalation stretch goal in scope for v1**, or should Phase
   2 build only the fixed-cheap-model path and leave escalation as a documented future
   direction in the writeup?

   let's go ahea and build the stretch goal since that reflects architectural and design maturity and this is as much an exhibit to accompany the portfolio / resume site as it is something built purely for function. 

## Note on category balance

The brief asked for "~35" and "~15" — landed at exactly 35 and 15. Easy-lookup / cross-page /
limitation-inclusive split is roughly 12/11/12 within the 35, which spans all three types the
brief named without being mechanically even (a few items, like G20 and G31, straddle
cross-page and limitation-inclusive — flagged where that happens rather than forcing a single
bucket). G36–G41 were added on 2026-09-08 by the claim audit and G42 on 2026-09-09, making it
12/11/19 within 42; the set is expected to grow this way whenever a published claim is retracted or
materially qualified, since a correction the eval can't detect is one the site can quietly regress
on.

G42 is the first item added for a correction that retracts **nothing**. The agentic-orchestration
page was accurate; what it had not disclosed was that its backing repository is private, so no
reader could check it. That makes G42 the template for *an unverifiable claim rather than a false
one* — it fails an answer that treats the page's file-level detail as independently confirmable, and
equally fails one that overcorrects into calling the claims false.