The Leak-Test — full guide (hand this to any AI helping on KAI)

_Two files, one idea. This doc explains what the tool is, why it exists, when to

reach for it, exactly how to call it, and — just as important — what it does

not do so nobody oversells it._

---

0. The one-paragraph version

The leak-test answers a single, sharp question: **"Does signal Y carry

information that a candidate set X cannot reconstruct?"** If X can rebuild Y, Y is

_redundant / derived_ (safe to merge, or to compute from X). If a chunk of Y

survives every attempt to rebuild it, that chunk is a real hidden channel and

Y must stay separate. It measures this as a residual — the fraction of Y left

unexplained — using both a linear fit and a non-linear held-out model, so a

non-linear identity isn't mistaken for a hidden channel. That's the whole

instrument. Everything else is hardening so the number can be trusted.

---

1. The two files

leak_universal imports leak_test_hardened for its non-linear engine, so

keep both files in the same folder. If leak_test_hardened (or scikit-learn) is

missing, leak_universal still runs — it drops to linear-only and says so in

the output. It never pretends it checked the non-linear channel when it didn't.

---

2. Reasons to use it (when to reach for this tool)

• "Are these two modules the same thing?" — merge decisions. (This is how #4

raphe → #11 serotonin got decided: 0.01% residual = duplicate.)

• "Is this output derived from something we already have?" — dedup, and

spotting a value that's just an EMA/lag of another (e.g. dopamine tonic).

• "Is this signal real, or a relabel?" — caught SCN consolidation_pressure

being a renamed phase (0.4% non-linear residual).

• "The comment claims X and Y are different — is that true in the CODE?" —

proved BNST threat_context is genuinely tonic, not a lagged amygdala

(103% residual vs the amygdala) — so the phasic/tonic split is real, not a story.

• "What is this residual — a pattern or noise?" — the SRHT stage

(leak_universal only). It found the hidden sine in a test signal that a plain

R² reported only as "unexplained."

• "Which input actually closes the gap?" — the sequential work-order scan

shows _where_ in an ordered candidate list the residual collapses.

Do not use it to prove two things are identical (it can only ever say

"redundant given THESE inputs"), and do not trust a merge verdict that rests

on hand-built synthetic data — the tool flags that and asks for real telemetry.

---

3. What it does NOT do (read this before quoting a result)

• A low residual ≠ "the same thing." It means _reconstructable from the

candidates you supplied_. Add a different candidate set, get a different answer.

• SRHT does not find "meaning." It is a fast random projection (see §7). It

measures whether leftover energy concentrates (structured) or spreads

(noise). "Emergence" here has a precise definition, not a mystical one.

• Synthetic data can be gamed by its own construction. A verdict from a trace

you hand-built proves the _code path_, not the _live system_. Re-run on

kai.exe --probe telemetry before executing any irreversible merge.

• Multi-method does not change arithmetic (see §8). 28/7 is 4. What changes is

which _structure_ is visible in which _framing_.

---

4. The pipeline (what leak_universal.analyze() runs)

Y, X  ─►  0 PROFILE      measure variance, autocorrelation, saturation, trend
          1 AUTO-CONFIG  derive settings from the profile (add lags? trust ratio?)
          2 MULTI-METHOD residual under levels / changes / cumulative / log
              │           └► CARRY-OVER = the residual + stats (the "next equation's data")
          3 GATE ◄── is the first pass already complete, or is there a carry-over
              │        worth building the SRHT equation for?
              │        ├─ SKIP → first answer stands (known-good / clean / low-var)
              │        └─ RUN  → hand the carry-over forward ↓
          3b SRHT        (only if the gate said RUN) is the leftover pattern or noise?
          4 VERDICT      one call, plus follow-up questions it spawns
          └► CHAIN       every stage logged as question → answer → next question

Stage 3b's input is stage 2's output (the residual vector). That is the

"the first answer builds the next equation" behaviour, made literal: the

unexplained part of the fit becomes the thing SRHT analyses — **but only when the

gate decides a carry-over exists.**

The conditional cascade (early-exit) — not everything needs a full SRHT

The pipeline is a cascade with an early exit, not a fixed straight line. The

first-pass answer (profile + multi-method residual) can be complete on its own.

The SRHT GATE decides whether to build the next equation at all:

• SKIP (first pass complete) for exactly two honest reasons: the case is

known-good (known_good=True — the caller asserts it's already verified), or

the signal is low-var/saturated (its residual can't be embedded — judge by

RMSE). For a known-good thing the pipeline never builds the SRHT stage.

• RUN (carry over) otherwise — including when levels *looks* redundant. SRHT is

O(n log n) cheap, and (once calibrated, below) it does not false-positive on

noise, so running it costs almost nothing and its 3-tier reading decides whether

a carry-over really existed.

first_pass_complete is then true only when there is genuinely no carry-over: the

gate skipped for an honest reason, or SRHT ran and read the residual as

noise. It is *not* set just because levels looked redundant.

> Why not short-circuit on "redundant in levels"? An earlier version did, on the

> reasoning that if the candidates explain the signal the residual is a scale

> artifact. A calibrated adversarial sweep disproved that. With

> y = 2x + A·sin(t/7) (the sine an independent hidden channel), at A = 0.12 the

> levels residual is 0.4% — "redundant" — yet the residual is a real sine that

> SRHT catches as STRUCTURED. Even non-flipping cases at A = 0.05 (0.2% residual)

> hide it. A small residual can be a real deterministic signal, so the tool no

> longer skips there. This is the project's "verify the verification" rule applied

> to the tool itself.

The result always exposes first_pass_complete (bool) and carry_over (the

residual + stats). The chain records the gate's decision and _why_. Pass

known_good=True to force the short-circuit for things you already trust.

---

5. How to use it (API + copy-paste examples)

The simplest call

from leak_universal import analyze
rep = analyze(Y, X, name="my_signal")
print(rep["verdict"])          # ORTHOGONAL / REDUNDANT / BORDERLINE / INCONCLUSIVE
print(rep["chain"].text())     # the readable question→answer chain

• Y : 1-D array — the signal you want to explain.
• X : 1-D or 2-D array — the candidate explainer(s). Columns = candidates.
• returns a dict: verdict, profile, config, multi_method, srht,

followups, and chain (a Chain object; call .text()).

Draw the graphs

rep = analyze(Y, X, name="my_signal", plot_path="my_signal.png")
# writes a 2-panel PNG: residual-by-method bars + SRHT residual spectrum

Sequential work-order scan (which input closes the gap?)

from leak_universal import sequential_scan
steps = sequential_scan(Y, [c1, c2, c3, c4], names=["c1","c2","c3","c4"])
for s in steps:
    print(s["added"], s["nonlinear"], s["verdict"])
# adds candidates ONE AT A TIME, in order; shows the residual after each

Force a setting (override the auto-config)

rep = analyze(Y, X, name="s", override={"use_lags": True, "merge_below": 8})

Short-circuit a known-good case (skip the SRHT stage)

rep = analyze(Y, X, name="already_verified", known_good=True)
print(rep["first_pass_complete"])   # True — SRHT never built
print(rep["carry_over"])            # still exposed, in case a later stage wants it

Just the raw number (bypass the adaptive layer)

from leak_test_hardened import leak, verdict
r = leak(Y, X)                 # {linear, nonlinear, rmse, var, flag}
print(verdict(r))

---

6. The rules / means it obeys (the guardrails it can tune WITHIN, not break)

These live at the top of leak_universal.py. The tool chooses which to apply

from the data profile; it cannot violate them. This is what "adaptive but bound to

rules" means.

Auto-config rules (derived, not guessed):

• stateful Y (high autocorrelation) → add lags so the model sees history;
• near-constant or saturated Y → stop trusting the ratio, judge by RMSE;
• otherwise → defaults.

Every choice it makes is written into the chain under PROFILE + AUTO-CONFIG, so

you can always see _why_ it configured itself the way it did.

---

7. The SRHT stage — what it is and what it's for

SRHT = Subsampled Randomized Hadamard Transform. It's the "old maths" the tool

leans on: Hadamard matrices (1893) and Walsh functions (1923), made into a fast

random projection by Ailon & Chazelle (2006). Mechanically:

P = sqrt(n/k) · S · H · D · M
   D = random ±1 sign flips      (spreads energy so no sample dominates)
   H = normalised Hadamard        (applied in O(n log n) via Fast Walsh–Hadamard)
   S = subsample k of n rows      (the compression)

It projects a big matrix into a small one while preserving the geometry

(distances/angles) with low distortion — cheaply. That's all it is: a fast, honest

squeeze.

Why the leak-test uses it. After the fit, the leftover (residual) is turned

into a delay-embedding matrix (sliding windows of the residual — a Hankel matrix),

SRHT-projected, and its singular spectrum measured — reported as concentration

(share of energy in the top third of components) and pr_normalized (participation

ratio, 0 = one mode, 1 = flat/noise). It returns a three-tier reading:

Why three tiers, and why these numbers — the calibration. An earlier version

used one loose cutoff and called pure noise "structured." A labeled bank (below,

stable across seeds) fixed the thresholds. pr_norm is the clean separator: real

deterministic channels sit at ≤ 0.12, every noise type at ≥ 0.25.

The binary yes/no would have to lie about the last row; the third tier tells the

truth. That is the "see what a single R² misses" part — measurable, calibrated, not

vibes. In the demo, Y = x + sin(t/7) reported 34% "unexplained" — SRHT showed

that 34% is one clean mode (the sine), tier = structured. (Period 7 — a nod to

the 28/7 example.)

---

8. The multi-method "28/7" idea — done honestly

You framed this as: _there are many ways to work a problem, and different workings

give different answers._ Here is the straight version, because you asked me not to

nod along:

• In ordinary arithmetic, 28 / 7 = 4. That value does not change with method.

Anyone who tells you 28/7 = 13 is not doing division. I'm not going to encode a

claim that it does.

• What genuinely changes with method is the DECOMPOSITION — how you _group_ and

_represent_ the computation. 28 as 7+7+7+7, as 4×7, as (30−2)/7, in base-2,

as a running total, as a rate of change — same number, different structure

exposed. And **for a time-series, the framing you test in decides what patterns

are visible.** That part of your intuition is correct and useful.

So the tool operationalises the _true_ half of the idea. Stage 2 runs the residual

test under four legitimate framings of the same data:

When the verdict flips between framings, that _is_ the finding. Example from

the demo: x + hidden_sine is ORTHOGONAL in levels but REDUNDANT in

changes — which tells you the hidden channel lives in the values, not the

motion. A single method would have hidden that. This is the "multidimensional

sequential math shift" — not different answers to 28/7, but different windows

onto the same object, each valid, compared side by side.

---

9. The provenance chain — how to read it

Every run builds a Chain: the root question never disappears, and each stage

appends {question, method, answer, next_question}. Print it with

rep["chain"].text(). You read it top-to-bottom as a conversation the tool had

with the data:

ROOT QUESTION: Does the candidate set explain 'Y' — or does 'Y' carry
               information the candidates cannot reconstruct?
[1] PROFILE + AUTO-CONFIG   Q: what kind of signal is this? → A: stateful, lags on
[2] MULTI-METHOD RESIDUAL   Q: how much is unexplained, and does it depend on framing?
[3] SRHT EMERGENCE          Q: is the leftover a pattern or noise?
[4] VERDICT                 A: the call + the new questions it spawns

The next field of each link is the actual input to the next stage — so the chain

is not a summary written afterwards, it's the real path the computation took. The

follow-up questions in stage 4 are the tool handing you the next thing to ask.

---

10. Requirements & graceful degradation

• Needs: numpy. Wants: scikit-learn (non-linear engine),

matplotlib (plots). Missing either → it runs and tells you what it skipped.

• Deterministic: SRHT uses a fixed seed by default, so runs reproduce.
• Cost: O(n log n) in the SRHT stage; comfortable to a few ×10⁴ samples.
• Files must sit together: leak_universal.py next to leak_test_hardened.py.

---

11. The honest bottom line

The leak-test is a residual detector with guardrails. Its power is that it

refuses to be fooled — by non-linear identities (the boosting stage), by saturated

signals (the low-var guard), by trending data (shuffled folds), by single-framing

blindness (multi-method), and by "unexplained = noise" laziness (SRHT). Its limit

is equally firm: it only ever speaks about the candidates you gave it, and a

synthetic verdict is a hypothesis until real telemetry confirms it. Used that way,

it's the instrument that lets math decide instead of opinion — which is the

whole point.

Sources for the methods:
  • Ailon & Chazelle (2006), "Approximate nearest neighbors and the fast
    Johnson–Lindenstrauss transform" — the SRHT.
  • Walsh (1923) / Hadamard (1893) — the transform's origins ("the old tool").
  • Gradient-boosted held-out residual + shuffled K-fold — the non-linear channel.