KAI — The Native-Voice Training Loop (word + listen correctly)

*Grounded in the real pipeline: runpod-training/, src/cognition/corpus_trainer.rs,

src/cognition/ingest_filter.rs, src/cognition/coherence.rs, RSHL-LLM-TRAINING-READINESS.md,

RUNPOD-TRAINING-HANDOFF.md. This is the loop that makes KAI word and listen correctly from

his OWN voice, so the LLM fallback fires less and less.*

---

The honest split: there are TWO trainings, not one

From your own readiness doc (RSHL-LLM-TRAINING-READINESS.md): **you cannot backprop the

lattice** — SparseVec::encode is a fixed, non-invertible hash with zero learnable parameters.

So "training KAI to talk" is really two different mechanisms, and keeping them separate is what

makes this tractable:

• Track A — WORD (gradient training). KAI's from-scratch ternary transformer is a real

trainable model (runpod-training/train_kai_transformer.py), and its inference path is now

live in Rust (kai_native_decode in language_warehouse.rs — a real forward+sample loop, no

longer the July "TODO scaffold"). This is the heavy fluency lever. RunPod / GPU.

• Track B — LISTEN + associative WORD (no backprop). The RSHL symbolic path learns by

*association and coverage*, not gradients: corpus_trainer binds (input→reply) pairs and

trains a light Response-MLP bias into build_generative_state; lexicon coverage decides which

words the symbolic decoder can even commit to; the ingest filter keeps the memory clean so

retrieval ("listening") surfaces the right thing. At home / overnight.

You need both. Track B is cheap and improves listening + the symbolic voice immediately; Track A

is the expensive fluency jump. Do them in the order below — never train on dirty data.

---

The loop (ordered — each step has a real artifact)

0. Precondition — turn on the meter (the build you're about to run)

KAI_NATIVE_PUBLIC_VOICE=1 KAI_VOICE_TELEMETRY=1 after the routing build. This makes KAI's own

voice answer AND logs, per turn, which voice answered + the coherence score. **Everything below

is measured by this meter.** Without it you're training blind.

1. Clean the corpus FIRST (do-first, low-risk, runnable now)

corpus_trainer learns from data/training_corpus/corpus_*.jsonl — every reply KAI logged. If

those contain scaffold artifacts, KAI trains on his own bad output (the Codex records exactly

this, v9.10.564). Run the cleaner I staged:

cd C:\KAI
python clean_training_corpus.py

Non-destructive: it writes *.clean.jsonl next to each file, drops pure artifacts

("Reasoning for…", "System Anchor:", "[MIRROR]"…), and salvages the real sentence out of

persona/sample wrappers ("Language sample (AI speaker Leo): Leo: …" → the sentence). Review

the .clean files, then point training at them. Same junk set the engine's ingest_filter

already rejects — now applied to the training corpus too.

2. Track B at home — lexicon + Response-MLP (cheap wins for listen + symbolic word)

• Grow lexicon coverage from trusted sources (your Google/Wikipedia + feedback loop). Feed

clean, well-formed sentences through ingest so the StatLexicon covers more words *with

correct usage* — the symbolic decoder can only commit to words it has encoded, and richer

coverage sharpens retrieval too.

• Retrain the Response-MLP (corpus_trainer mode 2) on the cleaned corpus so

build_generative_state biases toward historically-good replies.

• Verify memory hygiene: confirm ingest_filter gates the nightly overnight_pipeline so

no new junk becomes cells. (This is the "listen correctly" guard.)

3. Track A on RunPod — retrain the ternary transformer (the fluency jump)

Use the existing package. Point the text export at the cleaned corpus from step 1.

# local: rebuild the training text export from the CLEAN corpus, then upload
#   runpod-training/upload-ready/{kai-scripts.zip, kai-texts.zip, setup_on_pod.sh}
# on the A100 pod (per RUNPOD-TRAINING-HANDOFF.md):
bash setup_on_pod.sh
bash run.sh        # A100: batch 48, seq 512, ~5 epochs, dim 768 / 6 layers / 12 heads
# checkpoints: checkpoint_step{N}.pt + checkpoint_latest.pt  (touch SAVE_NOW to snapshot)

Known pod gotchas already solved in-repo: pod_fix_train_nan.py (NaN blowups),

pod_patch_resume.py (resume), split uploads kai-data-part-a{a..f} if Jupyter rejects the

tarball.

4. Install the trained model + restart + measure

Convert the checkpoint to the Rust load format and drop it in place:

models/kai-native/weights.bin      # ternary-packed weights
models/kai-native/config.json      # KaiNativeConfig (dim/layers/heads/vocab)

load_kai_native() auto-loads it on boot (KAI_NATIVE_MODEL default ON). Restart with the

flags, send the same battery, and read the meter: native score + accept per turn.

5. Calibrate the bar, then iterate on what still falls back

• Set coherence::min_score where native passes on genuinely good output and falls back on

genuinely bad — using the real scores from step 4, not a guess.

• Read the fallback log: the asks where native still loses to the LLM are your next training

targets (specific topics, longer/technical prompts). Feed those back into steps 1–3.

• Track the native accept-rate over rounds. The number climbing IS "KAI talking correctly,

from his own voice." When it's high enough, flip NATIVE_ONLY and the LLM becomes a true last

resort.

---

How this maps to your three words

• Listen → step 2 (lexicon/retrieval) + step 1/2 memory hygiene. Better recall = fewer

confident-but-wrong replies.

• Word → step 3 (transformer fluency) + step 2 (symbolic coverage + Response-MLP), gated by

the coherence critic so only good phrasing posts.

• Post → already handled by the routing + cleaners shipped this session; the meter proves it.

---

Honest confidence & risks

• Track B (clean + lexicon + MLP) is low-risk, cheap, and helps immediately — high confidence.
• Track A (RunPod retrain) is the real fluency lever but the expensive, iterative one — a

single retrain won't be perfect; the meter tells you how much more it needs. Medium confidence

on how many rounds; high confidence it's the right lever.

• The one thing that would stall this: training on the dirty corpus. Step 1 is non-negotiable and

first. Everything else is safe to iterate.

• RAM: running the native brain costs ~10GB (Codex note). If the box is tight, that's the

constraint to watch when native voice is on.

Bottom line: clean → grow lexicon + MLP (listen) → retrain the transformer (word) → install

• measure → target the misses. The coherence meter turns "talk correctly" from a feeling into a

number you drive up.