# The Leak-Test — full guide (hand this to any AI helping on KAI)

_Two files, one idea. This doc explains what the tool is, why it exists, when to
reach for it, exactly how to call it, and — just as important — what it does
**not** do so nobody oversells it._

---

## 0. The one-paragraph version

The leak-test answers a single, sharp question: **"Does signal Y carry
information that a candidate set X cannot reconstruct?"** If X can rebuild Y, Y is
_redundant / derived_ (safe to merge, or to compute from X). If a chunk of Y
survives every attempt to rebuild it, that chunk is a **real hidden channel** and
Y must stay separate. It measures this as a **residual** — the fraction of Y left
unexplained — using both a linear fit and a non-linear held-out model, so a
non-linear identity isn't mistaken for a hidden channel. That's the whole
instrument. Everything else is hardening so the number can be trusted.

---

## 1. The two files

| File | What it is | Use it when |
|---|---|---|
| `leak_test_hardened.py` | The **canonical v3** instrument. Fixed thresholds, hand-driven. Battle-tested across 24 KAI modules. | You want the exact, reproducible number for a specific module decision. This stays the reference. |
| `leak_universal.py` | The **adaptive** sibling. Profiles the data, configures itself, runs the residual test several ways, hands the leftover to SRHT, logs a readable chain. Plug anything in. | You want to point it at an unknown signal and have it figure out the right settings and surface structure a single number would miss. |

`leak_universal` **imports** `leak_test_hardened` for its non-linear engine, so
keep both files in the same folder. If `leak_test_hardened` (or scikit-learn) is
missing, `leak_universal` still runs — it drops to linear-only and **says so** in
the output. It never pretends it checked the non-linear channel when it didn't.

---

## 2. Reasons to use it (when to reach for this tool)

- **"Are these two modules the same thing?"** — merge decisions. (This is how #4
  raphe → #11 serotonin got decided: 0.01% residual = duplicate.)
- **"Is this output derived from something we already have?"** — dedup, and
  spotting a value that's just an EMA/lag of another (e.g. dopamine tonic).
- **"Is this signal real, or a relabel?"** — caught SCN `consolidation_pressure`
  being a renamed phase (0.4% non-linear residual).
- **"The comment claims X and Y are different — is that true in the CODE?"** —
  proved BNST `threat_context` is genuinely tonic, **not** a lagged amygdala
  (103% residual vs the amygdala) — so the phasic/tonic split is real, not a story.
- **"What is this residual — a pattern or noise?"** — the SRHT stage
  (`leak_universal` only). It found the hidden sine in a test signal that a plain
  R² reported only as "unexplained."
- **"Which input actually closes the gap?"** — the sequential work-order scan
  shows _where_ in an ordered candidate list the residual collapses.

**Do not** use it to prove two things are identical (it can only ever say
"redundant given THESE inputs"), and **do not** trust a merge verdict that rests
on hand-built synthetic data — the tool flags that and asks for real telemetry.

---

## 3. What it does NOT do (read this before quoting a result)

- A **low residual ≠ "the same thing."** It means _reconstructable from the
  candidates you supplied_. Add a different candidate set, get a different answer.
- **SRHT does not find "meaning."** It is a fast random projection (see §7). It
  measures whether leftover energy **concentrates** (structured) or **spreads**
  (noise). "Emergence" here has a precise definition, not a mystical one.
- **Synthetic data can be gamed by its own construction.** A verdict from a trace
  you hand-built proves the _code path_, not the _live system_. Re-run on
  `kai.exe --probe` telemetry before executing any irreversible merge.
- **Multi-method does not change arithmetic** (see §8). 28/7 is 4. What changes is
  which _structure_ is visible in which _framing_.

---

## 4. The pipeline (what `leak_universal.analyze()` runs)

```
Y, X  ─►  0 PROFILE      measure variance, autocorrelation, saturation, trend
          1 AUTO-CONFIG  derive settings from the profile (add lags? trust ratio?)
          2 MULTI-METHOD residual under levels / changes / cumulative / log
              │           └► CARRY-OVER = the residual + stats (the "next equation's data")
          3 GATE ◄── is the first pass already complete, or is there a carry-over
              │        worth building the SRHT equation for?
              │        ├─ SKIP → first answer stands (known-good / clean / low-var)
              │        └─ RUN  → hand the carry-over forward ↓
          3b SRHT        (only if the gate said RUN) is the leftover pattern or noise?
          4 VERDICT      one call, plus follow-up questions it spawns
          └► CHAIN       every stage logged as question → answer → next question
```

Stage 3b's input **is** stage 2's output (the residual vector). That is the
"the first answer builds the next equation" behaviour, made literal: the
unexplained part of the fit becomes the thing SRHT analyses — **but only when the
gate decides a carry-over exists.**

### The conditional cascade (early-exit) — not everything needs a full SRHT

The pipeline is a **cascade with an early exit**, not a fixed straight line. The
first-pass answer (profile + multi-method residual) can be **complete on its own**.
The `SRHT GATE` decides whether to build the next equation at all:

- **SKIP (first pass complete)** for exactly two honest reasons: the case is
  **known-good** (`known_good=True` — the caller asserts it's already verified), or
  the signal is **low-var/saturated** (its residual can't be embedded — judge by
  RMSE). For a known-good thing the pipeline **never builds the SRHT stage**.
- **RUN (carry over)** otherwise — including when levels *looks* redundant. SRHT is
  O(n log n) cheap, and (once calibrated, below) it does not false-positive on
  noise, so running it costs almost nothing and its 3-tier reading decides whether
  a carry-over really existed.

`first_pass_complete` is then true only when there is genuinely no carry-over: the
gate skipped for an honest reason, **or** SRHT ran and read the residual as
**noise**. It is *not* set just because levels looked redundant.

> **Why not short-circuit on "redundant in levels"?** An earlier version did, on the
> reasoning that if the candidates explain the signal the residual is a scale
> artifact. **A calibrated adversarial sweep disproved that.** With
> `y = 2x + A·sin(t/7)` (the sine an independent hidden channel), at `A = 0.12` the
> levels residual is **0.4% — "redundant"** — yet the residual is a real sine that
> SRHT catches as STRUCTURED. Even non-flipping cases at `A = 0.05` (0.2% residual)
> hide it. A small residual can be a real deterministic signal, so the tool no
> longer skips there. This is the project's "verify the verification" rule applied
> to the tool itself.

The result always exposes `first_pass_complete` (bool) and `carry_over` (the
residual + stats). The chain records the gate's decision and _why_. Pass
`known_good=True` to force the short-circuit for things you already trust.

---

## 5. How to use it (API + copy-paste examples)

### The simplest call
```python
from leak_universal import analyze
rep = analyze(Y, X, name="my_signal")
print(rep["verdict"])          # ORTHOGONAL / REDUNDANT / BORDERLINE / INCONCLUSIVE
print(rep["chain"].text())     # the readable question→answer chain
```
- `Y` : 1-D array — the signal you want to explain.
- `X` : 1-D or 2-D array — the candidate explainer(s). Columns = candidates.
- returns a dict: `verdict`, `profile`, `config`, `multi_method`, `srht`,
  `followups`, and `chain` (a `Chain` object; call `.text()`).

### Draw the graphs
```python
rep = analyze(Y, X, name="my_signal", plot_path="my_signal.png")
# writes a 2-panel PNG: residual-by-method bars + SRHT residual spectrum
```

### Sequential work-order scan (which input closes the gap?)
```python
from leak_universal import sequential_scan
steps = sequential_scan(Y, [c1, c2, c3, c4], names=["c1","c2","c3","c4"])
for s in steps:
    print(s["added"], s["nonlinear"], s["verdict"])
# adds candidates ONE AT A TIME, in order; shows the residual after each
```

### Force a setting (override the auto-config)
```python
rep = analyze(Y, X, name="s", override={"use_lags": True, "merge_below": 8})
```

### Short-circuit a known-good case (skip the SRHT stage)
```python
rep = analyze(Y, X, name="already_verified", known_good=True)
print(rep["first_pass_complete"])   # True — SRHT never built
print(rep["carry_over"])            # still exposed, in case a later stage wants it
```

### Just the raw number (bypass the adaptive layer)
```python
from leak_test_hardened import leak, verdict
r = leak(Y, X)                 # {linear, nonlinear, rmse, var, flag}
print(verdict(r))
```

---

## 6. The rules / means it obeys (the guardrails it can tune WITHIN, not break)

These live at the top of `leak_universal.py`. The tool **chooses which to apply**
from the data profile; it cannot violate them. This is what "adaptive but bound to
rules" means.

| Constant | Value | Meaning |
|---|---|---|
| `LOW_VAR` | `1e-3` | Below this variance, the %-ratio is untrustworthy → judge by RMSE. |
| `AR_STATEFUL` | `0.60` | Lag-1 autocorrelation above this ⇒ signal is stateful ⇒ auto-add lags. |
| `SAT_FRAC` | `0.50` | Fraction pinned at an extreme above this ⇒ saturated ⇒ distrust ratio. |
| `MERGE_BELOW` | `6.0%` | Non-linear residual below this ⇒ REDUNDANT / derived. |
| `SEPARATE_ABOVE` | `15.0%` | Above this ⇒ ORTHOGONAL / keep separate. |
| `EMERGE_STRUCT_PR` | `0.22` | SRHT: `pr_norm` below this (with high conc) ⇒ **structured**. |
| `EMERGE_STRUCT_CONC` | `0.88` | Top-third energy share required for **structured**. |
| `EMERGE_NOISE_CONC` | `0.70` | conc below this ⇒ **noise** (guard, below lowest real conc). |
| `EMERGE_NOISE_PR` | `0.55` | `pr_norm` above this ⇒ **noise** (primary noise axis). |
| `LAG_SET` | `(1,5,25)` | Which lags get stacked for stateful signals. |

**Auto-config rules** (derived, not guessed):
- stateful Y (high autocorrelation) → **add lags** so the model sees history;
- near-constant or saturated Y → **stop trusting the ratio**, judge by RMSE;
- otherwise → defaults.

Every choice it makes is written into the chain under `PROFILE + AUTO-CONFIG`, so
you can always see _why_ it configured itself the way it did.

---

## 7. The SRHT stage — what it is and what it's for

**SRHT = Subsampled Randomized Hadamard Transform.** It's the "old maths" the tool
leans on: Hadamard matrices (1893) and Walsh functions (1923), made into a fast
random projection by Ailon & Chazelle (2006). Mechanically:

```
P = sqrt(n/k) · S · H · D · M
   D = random ±1 sign flips      (spreads energy so no sample dominates)
   H = normalised Hadamard        (applied in O(n log n) via Fast Walsh–Hadamard)
   S = subsample k of n rows      (the compression)
```

It projects a big matrix into a small one **while preserving the geometry**
(distances/angles) with low distortion — cheaply. That's all it is: a fast, honest
squeeze.

**Why the leak-test uses it.** After the fit, the leftover (residual) is turned
into a delay-embedding matrix (sliding windows of the residual — a Hankel matrix),
SRHT-projected, and its singular spectrum measured — reported as `concentration`
(share of energy in the top third of components) and `pr_normalized` (participation
ratio, 0 = one mode, 1 = flat/noise). It returns a **three-tier** reading:

| tier | means | thresholds |
|---|---|---|
| **structured** | a real deterministic channel the candidates missed | `pr_norm < 0.22` **and** `conc ≥ 0.88` |
| **noise** | leftover is randomness; candidate set adequate | `pr_norm ≥ 0.55` **or** `conc < 0.70` |
| **partial** | real structure diluted by noise, or mere autocorrelation | anything in between |

**Why three tiers, and why these numbers — the calibration.** An earlier version
used one loose cutoff and **called pure noise "structured."** A labeled bank (below,
stable across seeds) fixed the thresholds. `pr_norm` is the clean separator: real
deterministic channels sit at ≤ 0.12, every noise type at ≥ 0.25.

| residual is… | conc | pr_norm | tier | correct? |
|---|---|---|---|---|
| gaussian noise (no channel) | 0.64 | 0.66 | noise | ✓ |
| uniform noise | 0.65 | 0.64 | noise | ✓ |
| AR(1) colored noise | 0.89 | 0.25 | partial | ✓ (correlated, not deterministic) |
| real sine (even A=0.05) | 1.00 | 0.06 | structured | ✓ |
| real sawtooth | 0.99 | 0.12 | structured | ✓ |
| 50/50 sine + noise | 0.75 | 0.29 | partial | ✓ (honest "look closer") |

The binary yes/no would have to lie about the last row; the third tier tells the
truth. That is the "see what a single R² misses" part — measurable, calibrated, not
vibes. In the demo, `Y = x + sin(t/7)` reported 34% "unexplained" — SRHT showed
that 34% is **one clean mode** (the sine), tier = structured. (Period 7 — a nod to
the 28/7 example.)

---

## 8. The multi-method "28/7" idea — done honestly

You framed this as: _there are many ways to work a problem, and different workings
give different answers._ Here is the straight version, because you asked me not to
nod along:

- **In ordinary arithmetic, 28 / 7 = 4.** That value does not change with method.
  Anyone who tells you 28/7 = 13 is not doing division. I'm not going to encode a
  claim that it does.
- **What genuinely changes with method is the DECOMPOSITION** — how you _group_ and
  _represent_ the computation. 28 as `7+7+7+7`, as `4×7`, as `(30−2)/7`, in base-2,
  as a running total, as a rate of change — same number, different structure
  exposed. And **for a time-series, the framing you test in decides what patterns
  are visible.** That part of your intuition is correct and useful.

So the tool operationalises the _true_ half of the idea. Stage 2 runs the residual
test under four legitimate framings of the same data:

| Framing | Question it exposes |
|---|---|
| **levels** | Do the candidates explain the value? |
| **changes** (first difference) | Do they explain the _dynamics_ / motion? |
| **cumulative** (running total) | Do they explain the _accumulation_? |
| **log** | Do they explain _multiplicative_ / proportional structure? |

When the verdict **flips** between framings, that _is_ the finding. Example from
the demo: `x + hidden_sine` is ORTHOGONAL in **levels** but REDUNDANT in
**changes** — which tells you the hidden channel lives in the values, not the
motion. A single method would have hidden that. This is the "multidimensional
sequential math shift" — not different answers to `28/7`, but different windows
onto the same object, each valid, compared side by side.

---

## 9. The provenance chain — how to read it

Every run builds a `Chain`: the **root question** never disappears, and each stage
appends `{question, method, answer, next_question}`. Print it with
`rep["chain"].text()`. You read it top-to-bottom as a conversation the tool had
with the data:

```
ROOT QUESTION: Does the candidate set explain 'Y' — or does 'Y' carry
               information the candidates cannot reconstruct?
[1] PROFILE + AUTO-CONFIG   Q: what kind of signal is this? → A: stateful, lags on
[2] MULTI-METHOD RESIDUAL   Q: how much is unexplained, and does it depend on framing?
[3] SRHT EMERGENCE          Q: is the leftover a pattern or noise?
[4] VERDICT                 A: the call + the new questions it spawns
```

The `next` field of each link is the actual input to the next stage — so the chain
is not a summary written afterwards, it's the real path the computation took. The
follow-up questions in stage 4 are the tool handing you the next thing to ask.

---

## 10. Requirements & graceful degradation

- **Needs:** `numpy`. **Wants:** `scikit-learn` (non-linear engine),
  `matplotlib` (plots). Missing either → it runs and tells you what it skipped.
- **Deterministic:** SRHT uses a fixed seed by default, so runs reproduce.
- **Cost:** O(n log n) in the SRHT stage; comfortable to a few ×10⁴ samples.
- **Files must sit together:** `leak_universal.py` next to `leak_test_hardened.py`.

---

## 11. The honest bottom line

The leak-test is a **residual detector with guardrails**. Its power is that it
refuses to be fooled — by non-linear identities (the boosting stage), by saturated
signals (the low-var guard), by trending data (shuffled folds), by single-framing
blindness (multi-method), and by "unexplained = noise" laziness (SRHT). Its limit
is equally firm: it only ever speaks about the candidates you gave it, and a
synthetic verdict is a hypothesis until real telemetry confirms it. Used that way,
it's the instrument that lets **math decide** instead of opinion — which is the
whole point.
```
Sources for the methods:
  • Ailon & Chazelle (2006), "Approximate nearest neighbors and the fast
    Johnson–Lindenstrauss transform" — the SRHT.
  • Walsh (1923) / Hadamard (1893) — the transform's origins ("the old tool").
  • Gradient-boosted held-out residual + shuffled K-fold — the non-linear channel.
```