# KAI — The Native-Voice Training Loop (word + listen correctly)

*Grounded in the real pipeline: `runpod-training/`, `src/cognition/corpus_trainer.rs`,
`src/cognition/ingest_filter.rs`, `src/cognition/coherence.rs`, `RSHL-LLM-TRAINING-READINESS.md`,
`RUNPOD-TRAINING-HANDOFF.md`. This is the loop that makes KAI word and listen correctly from
his OWN voice, so the LLM fallback fires less and less.*

---

## The honest split: there are TWO trainings, not one

From your own readiness doc (`RSHL-LLM-TRAINING-READINESS.md`): **you cannot backprop the
lattice** — `SparseVec::encode` is a fixed, non-invertible hash with zero learnable parameters.
So "training KAI to talk" is really two different mechanisms, and keeping them separate is what
makes this tractable:

- **Track A — WORD (gradient training).** KAI's from-scratch **ternary transformer** is a real
  trainable model (`runpod-training/train_kai_transformer.py`), and its inference path is now
  live in Rust (`kai_native_decode` in `language_warehouse.rs` — a real forward+sample loop, no
  longer the July "TODO scaffold"). This is the heavy fluency lever. RunPod / GPU.
- **Track B — LISTEN + associative WORD (no backprop).** The RSHL symbolic path learns by
  *association and coverage*, not gradients: `corpus_trainer` binds (input→reply) pairs and
  trains a light Response-MLP bias into `build_generative_state`; lexicon coverage decides which
  words the symbolic decoder can even commit to; the ingest filter keeps the memory clean so
  retrieval ("listening") surfaces the right thing. At home / overnight.

You need both. Track B is cheap and improves listening + the symbolic voice immediately; Track A
is the expensive fluency jump. Do them in the order below — **never train on dirty data.**

---

## The loop (ordered — each step has a real artifact)

### 0. Precondition — turn on the meter (the build you're about to run)
`KAI_NATIVE_PUBLIC_VOICE=1 KAI_VOICE_TELEMETRY=1` after the routing build. This makes KAI's own
voice answer AND logs, per turn, which voice answered + the coherence score. **Everything below
is measured by this meter.** Without it you're training blind.

### 1. Clean the corpus FIRST (do-first, low-risk, runnable now)
`corpus_trainer` learns from `data/training_corpus/corpus_*.jsonl` — every reply KAI logged. If
those contain scaffold artifacts, KAI trains on his own bad output (the Codex records exactly
this, v9.10.564). Run the cleaner I staged:

```
cd C:\KAI
python clean_training_corpus.py
```

Non-destructive: it writes `*.clean.jsonl` next to each file, **drops** pure artifacts
("Reasoning for…", "System Anchor:", "[MIRROR]"…), and **salvages** the real sentence out of
persona/sample wrappers ("Language sample (AI speaker Leo): **Leo:** …" → the sentence). Review
the `.clean` files, then point training at them. Same junk set the engine's `ingest_filter`
already rejects — now applied to the training corpus too.

### 2. Track B at home — lexicon + Response-MLP (cheap wins for listen + symbolic word)
- **Grow lexicon coverage** from trusted sources (your Google/Wikipedia + feedback loop). Feed
  clean, well-formed sentences through ingest so the `StatLexicon` covers more words *with
  correct usage* — the symbolic decoder can only commit to words it has encoded, and richer
  coverage sharpens retrieval too.
- **Retrain the Response-MLP** (`corpus_trainer` mode 2) on the **cleaned** corpus so
  `build_generative_state` biases toward historically-good replies.
- **Verify memory hygiene**: confirm `ingest_filter` gates the nightly `overnight_pipeline` so
  no new junk becomes cells. (This is the "listen correctly" guard.)

### 3. Track A on RunPod — retrain the ternary transformer (the fluency jump)
Use the existing package. Point the text export at the **cleaned** corpus from step 1.

```
# local: rebuild the training text export from the CLEAN corpus, then upload
#   runpod-training/upload-ready/{kai-scripts.zip, kai-texts.zip, setup_on_pod.sh}
# on the A100 pod (per RUNPOD-TRAINING-HANDOFF.md):
bash setup_on_pod.sh
bash run.sh        # A100: batch 48, seq 512, ~5 epochs, dim 768 / 6 layers / 12 heads
# checkpoints: checkpoint_step{N}.pt + checkpoint_latest.pt  (touch SAVE_NOW to snapshot)
```

Known pod gotchas already solved in-repo: `pod_fix_train_nan.py` (NaN blowups),
`pod_patch_resume.py` (resume), split uploads `kai-data-part-a{a..f}` if Jupyter rejects the
tarball.

### 4. Install the trained model + restart + measure
Convert the checkpoint to the Rust load format and drop it in place:

```
models/kai-native/weights.bin      # ternary-packed weights
models/kai-native/config.json      # KaiNativeConfig (dim/layers/heads/vocab)
```

`load_kai_native()` auto-loads it on boot (`KAI_NATIVE_MODEL` default ON). Restart with the
flags, send the same battery, and read the meter: native score + accept per turn.

### 5. Calibrate the bar, then iterate on what still falls back
- **Set `coherence::min_score`** where native passes on genuinely good output and falls back on
  genuinely bad — using the real scores from step 4, not a guess.
- **Read the fallback log**: the asks where native still loses to the LLM are your next training
  targets (specific topics, longer/technical prompts). Feed those back into steps 1–3.
- Track the **native accept-rate** over rounds. The number climbing IS "KAI talking correctly,
  from his own voice." When it's high enough, flip `NATIVE_ONLY` and the LLM becomes a true last
  resort.

---

## How this maps to your three words

- **Listen** → step 2 (lexicon/retrieval) + step 1/2 memory hygiene. Better recall = fewer
  confident-but-wrong replies.
- **Word** → step 3 (transformer fluency) + step 2 (symbolic coverage + Response-MLP), gated by
  the coherence critic so only good phrasing posts.
- **Post** → already handled by the routing + cleaners shipped this session; the meter proves it.

---

## Honest confidence & risks

- Track B (clean + lexicon + MLP) is **low-risk, cheap, and helps immediately** — high confidence.
- Track A (RunPod retrain) is the **real fluency lever but the expensive, iterative one** — a
  single retrain won't be perfect; the meter tells you how much more it needs. Medium confidence
  on how many rounds; high confidence it's the right lever.
- The one thing that would stall this: training on the dirty corpus. Step 1 is non-negotiable and
  first. Everything else is safe to iterate.
- RAM: running the native brain costs ~10GB (Codex note). If the box is tight, that's the
  constraint to watch when native voice is on.

**Bottom line:** clean → grow lexicon + MLP (listen) → retrain the transformer (word) → install
+ measure → target the misses. The coherence meter turns "talk correctly" from a feeling into a
number you drive up.