# RSHL → LLM Training-Readiness Plan

**Status:** DESIGN / READINESS ONLY — no training code written, no engine touched.
**Date:** 2026-07-04 · **Author:** engineering pass (read-only investigation)
**Rev 2 (2026-07-04):** added two owner refinements — **§6 CLOUD COMPUTE** (pretrain heavy in
the cloud, adapt light at home) and **§7 CONTINUAL / LIFELONG LEARNING** (the "open part" —
replay-based overnight consolidation of the LLM brain module, wired into KAI's *existing* sleep/
hippocampus loop). §8 revises the architecture, phases, risks and open decisions accordingly.
**Scope of this doc:** map the *real* current code, then specify exactly what it would take
to make KAI's RSHL lattice *trainable like an LLM* (next-token prediction with backprop),
so the owner can green-light a build. This is grounded in files that exist today — every
claim below cites a real path.

> **One-line honest summary:** KAI's RSHL core is a *fixed hash-based feature + associative
> memory*, not a gradient-trainable network. You cannot "backprop the lattice" as-is. But you
> already own every other piece needed to train a small LLM (a BPE tokenizer, a Candle
> transformer implementation, 100M+ tokens of corpus, and CUDA-capable Candle). The realistic
> path is: **add a small, genuinely backprop-trained language model, and keep RSHL as the
> feature/memory/retrieval substrate around it** — not to make the 16384-dim ternary lattice
> itself differentiable.

---

## 0. TL;DR for the owner

- **Can we "train RSHL to be an LLM"?** Not literally — the RSHL vector is produced by a
  *fixed, non-invertible hash function* (`SparseVec::encode`, `src/core/sparse_vec.rs:308`).
  It has **zero learnable parameters** and you can't decode tokens back out of it, so it can't
  be the *output* layer of a language model.
- **Recommended architecture: (c) hybrid, staged.** Start with **(a)** — a small trainable
  next-token *head* on top of frozen RSHL features (proves the training loop cheaply, honors
  "keep the RSHL core"). Then grow into **(c)** — a small trainable transformer LM with RSHL
  used as a kNN memory (retrieval-augmented). Do **not** pursue **(b)** (making the lattice ops
  differentiable) as the primary path — high research risk, low payoff on this hardware.
- **The single most important finding:** you already have a **full BitNet/LLaMA transformer in
  Candle** (`src/cognition/bitnet_llama.rs`) and a **generation loop with logits + sampling**
  (`src/cognition/language_warehouse.rs:474-598`) — but it is **inference-only, loaded from a
  frozen GGUF**. There is **no training loop, no loss, no optimizer, no autodiff anywhere** in
  the current "training" path. Today's "training" (`overnight_pipeline.py`,
  `distill_from_bitnet.py`) is *memory ingestion* (binding prompt→answer pairs into the
  lattice), **not** gradient descent.
- **Hardware reality (blunt):** a single RTX 4050 Laptop (6 GB VRAM) can train a **small**
  transformer from scratch (~5–30M params, char- or small-BPE, context 256–512) on his corpus
  in hours-to-days. It **cannot** train a 2B model from scratch, and shouldn't try. Scope =
  *small LM, RSHL-augmented, trained on his own data.*

---

## 1. Honest framing — what "train RSHL to be an LLM" can and cannot mean

### 1.1 What RSHL actually is (grounded)

The RSHL core is a **Vector Symbolic Architecture / hyperdimensional computing** substrate:

- **Representation:** `SparseVec` — a 16384-dim ternary vector, values in {−1, 0, +1}, stored
  sparse (`nz: Vec<u16>` indices + `vals: Vec<i8>`), default **~4% density** (~655 active dims;
  `SPARSITY = 0.04`, `DIM = 16384`, `src/core/sparse_vec.rs:18-22`).
- **Text → vector:** `SparseVec::encode` (`sparse_vec.rs:308`) is a **fixed hash** — it sums
  weighted hash contributions from char trigrams, normalized words (weight ×3, proper nouns ×6),
  word-pairs (some order info), char bigrams, and char 4-grams, then keeps the top-|magnitude|
  ~655 dims as ±1. **No learned weights. Deterministic. Many-to-one (not invertible).**
- **Ops:** `bind` = element-wise ternary multiply, self-inverse (`sparse_vec.rs:757, 784`);
  `bundle`/`superpose_sparse` = add + re-sparsify (`:576, :729`); `cosine` = retrieval metric
  (`:500`); cleanup via nearest-stored-vector search across the `Universe` lattice.
- **"Learning" today:** associative — you *store* vectors and *bundle/bind* them; recall is
  nearest-neighbor by cosine. The one genuinely adaptive component is `ResponseMlp`
  (`src/cognition/corpus_trainer.rs:250-370`): a single-hidden-layer sparse net updated by
  **Hebbian reward rules** (`train_step`, `hebbian_update`), **not** backprop.

### 1.2 Why the lattice can't just "be" an LLM

A next-token LLM needs, at minimum: (1) a differentiable map from context → a distribution over
a fixed vocabulary, and (2) gradients that flow back to update parameters to minimize
cross-entropy. RSHL fails both natively:

- **No parameters to train.** `encode` is a hash; there's nothing to differentiate.
- **No decodable output space.** You can't recover a token id from a bundled ternary vector; the
  hash is lossy and collides by design. So RSHL can be an *input/context* representation or a
  *memory*, never the softmax *output* head.
- **Non-differentiable ops.** `bind`/`bundle`/threshold-sparsify are hard/argmax operations;
  no gradient path.

**Conclusion:** "make RSHL trainable like an LLM" = *attach a real trainable next-token model to
the system and let RSHL feed / augment it.* The RSHL core stays intact (per project hard rules);
we add a parallel, trainable, Python/Candle-side model.

### 1.3 The three candidate architectures

**(a) Trainable next-token HEAD on frozen RSHL features.**
RSHL = frozen feature extractor + memory. A small trainable head does softmax next-token
prediction: `token_{t+1} = Head( RSHL_features(context ≤ t) )`.
- Pros: tiny, cheap, provable in a day; directly honors "keep the RSHL core, add a trainable
  LM head"; overfit-a-batch sanity check is trivial.
- Cons: RSHL bag-of-ngrams loses most word *order*, so a pure-RSHL-context head has a low fluency
  ceiling. Good as a **proof of the training loop and as a memory/bias signal**, not as a
  fluent generator on its own.

**(b) Differentiable relaxation of the lattice ops.**
Replace ternary/threshold ops with soft (real-valued, temperature-softmax) analogues so gradients
flow through a "learned encode."
- Pros: intellectually the "train the lattice itself" answer.
- Cons: **research project, not an engineering task.** You'd effectively rebuild the encoder as a
  neural net, losing the sparsity/energy properties that make RSHL interesting, and it's the
  worst fit for a 6 GB laptop. **Not recommended as the primary path.**

**(c) Hybrid: small trainable transformer LM + RSHL retrieval/memory (RECOMMENDED).**
Train a *small real language model* (its own learned token embeddings + a few transformer/MLP
blocks) on his corpus. Use RSHL as an external **kNN memory** (kNN-LM / RAG style): at each step,
retrieve the nearest stored continuations from the lattice and interpolate their next-token
evidence with the transformer's softmax.
- Pros: this is *actually* "developing an LLM" the normal way, at a size the 4050 can handle; it
  **reuses what you already have** (the Candle transformer scaffold + `language_warehouse`
  retrieval); RSHL stays central as memory + feature + retrieval, satisfying "keep the RSHL core,
  not just softmax."
- Cons: two components to wire (LM + retrieval interpolation). Manageable, and staged below.

**Recommendation: build (a) first as Phase 0 (loop proof), then evolve into (c).** Skip (b).

### 1.4 Data flow (words, not code)

```
his corpus (jsonl)  ──►  clean/dedup  ──►  tokenizer (Llama-3 BPE, EXISTS: models/tokenizer.json)
        │                                        │
        │                                   token id stream ──► shards (.bin of u16/u32 ids)
        ▼                                        │
  RSHL encode (FIXED)                            ▼
  context → SparseVec  ───────────────►  Trainable model
        (feature/memory)                  ┌──────────────────────────────────────────┐
                                          │ Phase 0 (a): Head(RSHL_feats) → softmax   │
                                          │ Phase 1+ (c): small Transformer LM        │
                                          │            + kNN-LM interpolation w/ RSHL  │
                                          └──────────────────────────────────────────┘
                                                         │
                                     next-token logits ──► cross-entropy loss vs true next id
                                                         │
                                          AdamW backward/step (PyTorch or Candle autodiff)
                                                         │
                                          checkpoints (.safetensors) ──► eval (perplexity /
                                                         │                next-token top-1/top-5)
                                                         ▼
                          plug back into KAI: LM logits + RSHL kNN evidence bias the existing
                          generate() sampler in language_warehouse.rs (the plumbing already exists)
```

---

## 2. Readiness checklist — EXISTS vs must BUILD

Each item is mapped to real files.

### (1) Training corpus + data pipeline
- **EXISTS (data):**
  - `data/harvest.jsonl` — **1,388,700 lines / 301 MB**, schema `{"text", "region", "source",
    "strength"}` (general knowledge statements, wikipedia-swarm etc.). ~**60–80M BPE tokens**.
  - `data/training_corpus/bulk_conversations.jsonl` — **103 MB**, schema `{"text"}` (reasoning/
    conversation prose). ~**20–25M tokens**.
  - `data/training_corpus/corpus_discord_YYYYMMDD.jsonl` — **his own KAI/Discord data**, schema
    `{"input", "reply", "state", "timestamp", ...}` — real prompt→reply **pairs** (ideal for
    instruction/response tuning). Tens of MB across days. ~**5–10M tokens**.
  - `data/bulk.jsonl` (7.5 MB, 7,406 lines), `data/kai-transcript.jsonl` (579 lines),
    `data/kai-texts.bin` (768 KB), `data/dictionary.json` (24 MB), `data/epistemic-rejections.jsonl`.
  - **Rough budget: ~100M+ tokens raw available; his *own* conversational data ~5–10M tokens.**
    Plenty for a small LM; his-own-data-only is enough for a persona/style model, borderline for
    from-scratch general fluency (mix in harvest/bulk for that).
- **BUILD:** a `data_prep` script: source jsonl → strip noise (reuse the noise filter idea in
  `corpus_trainer.rs:147 is_noisy`) → dedup → normalize → write a single clean text/id stream →
  shard to `.bin`. Nothing like this exists on the *gradient-training* side yet.
- **HONEST NOTE:** `data/transcripts.db` is **0 bytes (empty)** — despite the name, the real
  conversational corpus lives in `data/training_corpus/*.jsonl`, not a SQLite DB. Don't build the
  pipeline around `transcripts.db`.

### (2) Tokenizer / vocab
- **EXISTS:** `models/tokenizer.json` (**1.9 MB, Llama-3 BPE, vocab 128,256**) and
  `models/BitNet/tokenizer.json`. The Candle transformer config already assumes this vocab
  (`bitnet_llama.rs:41 vocab_size: 128256`). Loadable in Python via HF `tokenizers`, in Rust via
  the `tokenizers` crate.
- **DECISION, not a build:** for the Phase-0 POC, a **128k BPE head is wasteful** (huge softmax on
  6 GB). Recommend either **char-level** (vocab ~100) or a **small custom BPE (4k–8k)** trained on
  his corpus for the small-LM phases; keep the 128k Llama tokenizer only if/when you want the LM
  to interoperate with BitNet tokens. So: tokenizer *exists*; a *small* vocab is a 1-line
  `tokenizers` training step to build.

### (3) Trainable model definition
- **EXISTS (reusable scaffolding, inference-only today):**
  - `src/cognition/bitnet_llama.rs` — a **complete LLaMA-family forward pass in Candle**
    (RoPE, GQA `repeat_kv`, causal mask, RMSNorm, KV-cache; config hidden 2560 / 30 layers /
    vocab 128256). This is a *reference implementation* to shrink, **not** a thing to train at
    2B scale.
  - `src/cognition/language_warehouse.rs` — sparse ternary word embeddings extracted from BitNet
    + a real generation loop.
  - `Cargo.toml:64-66, 90` — `candle-core/nn/transformers 0.10.2` already deps, with an
    `llm-cuda` feature (`candle-*/cuda`). **Candle supports autodiff** (`Var`, `.backward()`,
    `candle_nn::AdamW`), so a training loop can live in Rust.
- **BUILD:**
  - Phase 0 (a): `RSHLHead` — e.g. `Linear(16384 → hidden) → GELU → Linear(hidden → vocab)`, or a
    2–4 layer MLP. **~1–20M params** depending on vocab. Input = mean/któ-pooled RSHL features of
    the context window.
  - Phase 1+ (c): `TinySLM` — learned token embedding (dim 256–512) + 2–6 transformer blocks +
    tied output head. **~5–30M params.** Plus a kNN-LM interpolation module that queries the RSHL
    `Universe` for nearest continuations.
- **Framework recommendation:** **prototype in Python/PyTorch** (fastest iteration, `tokenizers`,
  `datasets`, mature AMP/checkpointing, easy overfit-a-batch). **Port the winning small model to
  Candle** for in-engine integration later (Candle is already wired and CUDA-capable, so the
  trained `.safetensors` loads natively next to `bitnet_llama.rs`).

### (4) Training loop
- **EXISTS:** nothing gradient-based. Current `overnight_pipeline.py` / `distill_from_bitnet.py`
  do **memory ingestion / distillation-by-recall** (bind prompt→answer into the lattice); grep
  confirms **no `torch`, no `autograd`, no `loss.backward`, no optimizer** in the Python
  pipeline. `ResponseMlp::train_step` is Hebbian, not SGD.
- **BUILD:** standard next-token loop — batches of `(context_ids, target_id)` → forward → **cross-
  entropy** → AdamW → grad-clip → cosine LR w/ warmup → checkpoint `.safetensors` every N steps →
  resume. Add gradient accumulation + fp16/bf16 AMP to fit 6 GB. ~150 lines PyTorch.

### (5) Eval harness
- **EXISTS (adjacent, wrong metric):** `benchmarks/rshl_vs_llm.mjs`, `rshl_fullbench.mjs`,
  `rshl_scale_sweep.mjs` measure **energy / latency / cost per token**, and explicitly state
  *"RSHL RETRIEVES, LLM GENERATES … not capability equivalence"* (`rshl_vs_llm.mjs:27-29`).
  **They do not measure language quality (perplexity / next-token accuracy).**
- **BUILD:** a `lm_eval` harness computing **perplexity** and **next-token top-1/top-5 accuracy**
  on a held-out split. Wire it to sit *beside* `rshl_vs_llm` so you can report *quality* next to
  the existing *cost* numbers (quality-per-joule becomes a real comparison for the first time).

### (6) Compute / runtime plan
- **Hardware:** RTX 4050 Laptop — **6 GB VRAM**, ~194 GB/s bandwidth, Ada tensor cores. Plus the
  laptop CPU (already the RSHL substrate).
- **Feasible:** from-scratch small transformer (5–30M params, ctx 256–512, bf16, batch via grad-
  accum) trains at ~thousands of tokens/sec; **one epoch over ~10–50M tokens in hours**, a decent
  small model in **1–3 days** of wall time. LoRA-fine-tuning a small (<1B) model also fits.
- **NOT feasible (be blunt):** training the 2B BitNet from scratch (weeks–months, needs ≫6 GB);
  full fine-tune of any multi-B model; 128k-vocab softmax on long context at batch. Don't attempt.
- **Toolchain:** **PyTorch + CUDA for R&D**, Candle for the eventual in-engine port. Keep training
  a **separate offline process** (same discipline as `distill_from_bitnet.py`) — never in the
  live serving engine, never sets `KAI_NATIVE_BRAIN`.

### (7) Plug-back into KAI
- **EXISTS:** the integration seam is already there — `language_warehouse.rs:474-598` runs
  `forward → logits → apply bias → sample(temp, top-p, repeat-penalty) → next_token`, and it
  *already* biases logits from lattice evidence (`:584 logits_vec[token_id] += bias`). A trained
  head/LM drops into exactly this point.
- **BUILD:** load the trained `.safetensors`; either (a) use its logits as the `bias` source, or
  (c) replace/interpolate the `forward` logits with the small LM's + RSHL kNN evidence. Because
  the sampler plumbing exists, plug-back is small once a checkpoint exists.

**Checklist headline:** *Tokenizer ✔, corpus ✔ (~100M tokens), transformer scaffold ✔ (Candle,
inference-only), CUDA ✔, integration seam ✔. Must build: the gradient-training half — data-prep/
shard script, a small trainable model def, a next-token training loop with autodiff, and a
perplexity/accuracy eval harness. None of these exist today.*

---

## 3. Phased path to the first training run

### Phase 0 — Prove the loop (architecture (a), ~2–4 days)
**Goal:** a real backprop next-token model runs end-to-end on his data, overfits a single batch.
- **Build:** char-level tokenizer (or 4k BPE) over a small slice of `corpus_discord_*.jsonl`;
  `RSHLHead` (`Linear(16384→512)→GELU→Linear(512→vocab)`); PyTorch train loop; cross-entropy.
- **Verify (the classic sanity check):** **overfit a single batch to ~0 loss.** If it can't,
  the loop is wrong — fix before scaling. Then confirm held-out perplexity beats a unigram
  baseline.
- **Effort:** low. This is the green-light gate.

### Phase 1 — Small real LM on his corpus (architecture (c) core, ~1–2 weeks part-time)
**Goal:** a small transformer that generates his-style text with falling perplexity.
- **Build:** data-prep/shard pipeline over `corpus_discord` + a mixed slice of `harvest`/`bulk`;
  8k BPE; `TinySLM` (emb 384, 4 blocks, ctx 512, ~15–25M params); AMP + grad-accum + checkpoint/
  resume; `lm_eval` (perplexity + top-1/5).
- **Verify:** perplexity curve down on held-out; qualitative samples; compare energy/quality via
  the existing `rshl_vs_llm` harness extended with quality.
- **Effort:** medium.

### Phase 2 — RSHL retrieval augmentation (the "keep RSHL central" payoff, ~1 week)
**Goal:** kNN-LM — interpolate the small LM's softmax with next-token evidence retrieved from the
RSHL `Universe`.
- **Build:** a retrieval step that RSHL-encodes the context, finds nearest stored continuations,
  and blends their token evidence with LM logits (`p = λ·p_LM + (1−λ)·p_RSHL`); tune λ.
- **Verify:** perplexity drops vs Phase 1 LM alone, *especially* on his-own-domain prompts.

### Phase 3 — Plug back + optional Candle port (~1 week)
**Goal:** KAI's live language actually improves.
- **Build:** load checkpoint at the `language_warehouse.rs` sampler seam; optionally port `TinySLM`
  to Candle so it loads natively beside `bitnet_llama.rs`; keep it flag-gated + offline-trained.
- **Verify:** owner A/Bs KAI's replies with the LM on/off.

---

## 4. Open decisions for the owner

1. **Architecture:** confirm **(a)→(c) staged** (recommended) vs insisting on **(b)** differentiable
   lattice (not recommended — research risk, poor 6 GB fit).
2. **Tokenizer granularity:** **char-level** (simplest POC) vs **small custom BPE 4k–8k**
   (recommended for Phase 1) vs reuse **Llama-3 128k** (only if BitNet-token interop matters).
3. **Data policy:** **his-own-data-only** (persona/style, ~5–10M tokens — coherent "Kai voice",
   weaker general fluency) vs **mix in harvest/bulk** (~100M tokens — more fluent, less purely
   "him"). Recommended: his-data-heavy + a harvest/bulk minority for grammar.
4. **Framework/where it lives:** **PyTorch offline for R&D** then **Candle port** (recommended),
   vs train directly in Candle/Rust from the start (slower iteration, but no port and Candle is
   already wired with `llm-cuda`).
5. **RSHL's role:** frozen feature only (a) vs feature **+ kNN memory** (c, recommended) — i.e.
   how central RSHL stays in the trained system.

---

## 5. What this plan explicitly does NOT claim
- It does **not** claim we can train a GPT-scale or even 1B model on this laptop. Scope is a
  **small** LM (≤~30M params), RSHL-augmented.
- It does **not** claim the existing pipeline already trains anything by gradient — it doesn't;
  it ingests memory.
- It does **not** propose editing the live engine, `kaiverse.js`, `oracle.html`, or the RSHL core
  ops. Training is an **offline, additive** track, consistent with `distill_from_bitnet.py`'s
  safety model.

---

## 6. REFINEMENT 1 — Cloud compute (heavy pretrain in the cloud, light adaptation at home)

The 4050-only framing above (§2.6) was deliberately conservative. The owner has **cloud** —
a VPS / Google Cloud / rented cloud GPU or TPU — so the *hardware ceiling moves*. This does
**not** change the honest architecture (RSHL is still not backprop-trainable; §1 stands); it
changes **where the one big compute cost is paid**.

### 6.1 The clean split (the key idea)

There are two very different compute jobs, and they want two different machines:

- **HEAVY, one-time PRETRAIN — do it in the cloud.** Learning grammar/world-structure from
  scratch is the expensive part: many passes over billions of tokens. This is embarrassingly
  well-suited to a rented A100/L4/TPU for a day or two, then **you never pay it again**.
- **LIGHT, ongoing ADAPTATION — do it at home on the 4050.** Once you own good base weights,
  keeping KAI current with *his* lived experience is cheap: small LoRA/fine-tune deltas over
  thousands (not billions) of tokens. This is exactly what a 6 GB laptop is fine at, and it's
  what the continual-learning loop in §7 needs.

> **Rule of thumb: pretrain is a capex you rent once; consolidation is an opex you run at home.**
> Bring the cloud-trained `.safetensors` home, and from then on the 4050 only ever does small
> bounded updates + inference. Phase 0 (the proof-of-concept, §3) still runs **entirely local** —
> you don't rent anything until the loop is proven.

### 6.2 What becomes feasible with cloud

- **A genuinely coherent small base model.** On a single **A100 80 GB (~$1.0–1.4/hr on-demand,
  ~$0.6/hr spot)** or an **L4/L40S (cheaper, slower)**, you can pretrain a **~30M–125M-param**
  transformer from scratch on a Chinchilla-ish token budget (roughly 20 tokens/param → ~0.6B–2.5B
  tokens) in **hours, not weeks**. For calibration, Karpathy's llm.c reproduces **GPT-2 124M**
  (10B tokens) on an 8×A100 node in a few hours for **tens of dollars**; a single-GPU, smaller-
  token run of a 30–125M model is the same order of magnitude.
- **Ballpark cost/time (order-of-magnitude, plan with a 2–3× cushion for failed runs + tuning):**
  - *Phase-1 target, 50–125M params, ~1–3B tokens, single A100 spot:* ~**10–30 GPU-hours**,
    ~**$15–60** raw compute; call it **~$50–200 all-in** with experimentation.
  - *A stretch 350M model:* days on one A100 or a small multi-GPU node, **low-hundreds of $**.
  - *TPU alternative:* a **v5e/preemptible TPU** slice on Google Cloud trains small models very
    cost-effectively if you use JAX/Flax; more toolchain friction than PyTorch-on-A100.
- **NOT worth it:** renting to train multi-billion-param models from scratch (thousands of $,
  no payoff for KAI's purpose). The BitNet 2B stays a *frozen teacher*, never a from-scratch target.

### 6.3 Recommended concrete cloud path (Phase-1 pretrain)

- **Provider/instance:** a GPU-first host (**RunPod / Lambda / Vast.ai**) renting **1× A100 80 GB**
  (on-demand for a clean run, spot/interruptible if you checkpoint often). A100 80 GB removes the
  6 GB VRAM straitjacket entirely for a ≤125M model — big batches, longer context, fast epochs.
  Google Cloud is the choice if you specifically want TPU or want the VPS + storage in one place.
- **Framework:** **PyTorch + a nanoGPT-class trainer** (fastest path, mature AMP/checkpointing,
  `tokenizers`/`datasets`). Output **`.safetensors`**. Keep the model definition deliberately
  Candle-portable (plain LLaMA-ish blocks) so it later loads next to `bitnet_llama.rs` in-engine.
- **Data:** the corpus already inventoried in §2.1 (`data/harvest.jsonl` + `bulk_conversations.jsonl`
  for grammar/world-structure, `corpus_discord_*.jsonl` for "Kai voice"). Upload the sharded `.bin`
  once; train; download weights.
- **Deliverable home:** one `.safetensors` base checkpoint (+ tokenizer) copied to `C:\KAI\models\`.
  From here on, everything is local (§7).

---

## 7. REFINEMENT 2 — Continual / lifelong learning (the "open part")

This is the core of the owner's idea: KAI's small LLM **brain module** should *keep learning over
time* — "an open part for real-time learning… consolidation over time like how humans and living
creatures do." That is precisely **Complementary Learning Systems (CLS)** theory, and — the finding
that matters — **KAI already implements the CLS memory half in real Rust today.** The LLM piece
plugs into machinery that already exists; we are not inventing the biology, we are adding a cortex.

### 7.1 CONFIRMED: KAI's biological consolidation machinery already exists (real code)

The telemetry line *"Hippocampus CA3/CA1: Short-Term Patterns | Queued for Sleep Replay | Graduated
to Universe"* is **not decoration — it is a live loop.** Verified this pass:

- **Fast episodic memory (hippocampus) — `src/cognition/hippocampus.rs`.** A CA3 autoassociative
  pattern bank (`CA3_CAPACITY = 256`, `hippocampus.rs:44`), a CA1 recent-pattern buffer
  (`CA1_BUFFER = 12`, `:47`), DG-style **pattern separation** (`separate()`, `:288`) and CA3
  **pattern completion** (`complete()`, `:229`). Each stored trace carries `strength`,
  `survival_count`, `emotional_charge`, `region`, `source` (`HippocampalPattern`, `:65-91`).
  **This is one-shot learning:** a single Ryan turn is bound immediately —
  `self.engine.hippocampus.store(&input, …)` at **`main.rs:4741`**, alongside `episodic.store`.
- **The consolidation / "graduation" step — `hippocampus.rs:367 consolidate_into_universe()`.**
  Three biological gates: **Gate 1** strength threshold (0.55 neutral / 0.45 emotional, `:395`);
  **Gate 2** novelty (skip if Universe already knows it, boosted cosine > 0.65 → *reinforce* instead
  of re-store, `:419`); **Gate 3** survival ≥ 2 cycles unless emotional fast-track ≥ 0.60 (`:405`).
  Promoted traces are **removed from hippocampus and written into the Universe** (`universe.store`,
  `:443`) — literally *"Graduated to Universe."*
- **The driver loop — `main.rs:890-929`.** Every 50 ticks (~4 min): `hippocampus.decay()` then
  `consolidate_into_universe(&mut universe, coherence)`. A **spiral-coherence gate** (`tau_r < 0.35`
  suppresses consolidation, `hippocampus.rs:372`) mirrors stress-impaired memory transfer. Real
  stats are persisted to `data/hippocampus_status.json` (`main.rs:914`) — that's the dashboard readout.
- **The sleep/replay scheduler — `src/cognition/sleep.rs`.** Clock-gated to ~3:05 AM
  (`should_sleep()`, `:147`), it runs **NREM** (scan episodic for high salience×vividness) → **SWS**
  (consolidate top events, global synaptic **downscale ×0.92**, prune weak cells) → **REM**
  (recombine top events into novel associations) → wake report (`run_cycle()`, `:177-260`).
- **The nightly ingestion job — `overnight_pipeline.py`.** Confirmed **memory ingestion, NOT
  gradient training:** it scrapes/synthesizes study material, uses a teacher LLM (Groq/OpenRouter/
  Gemini or the offline **BitNet** teacher) to quiz/grade KAI, distills BitNet→lattice via
  `bulk-ingest`, and refreshes the ANN index. It even **stops cleanly at ~4:30 AM
  (`PIPELINE_STOP_HOUR`, `:106`) specifically so the engine-side sleep/consolidation can run
  without new material flooding in.** No `torch`, no `loss.backward`, no optimizer anywhere.

**Bottom line of the audit:** KAI already has the *hippocampus (fast) + Universe (slow) + sleep-
replay (transfer)* CLS triad — but the "slow store" is the **RSHL Universe lattice**, which (per §1)
is associative, not a trainable cortex. **The continual-learning upgrade = add the LLM brain module
as a second slow store, and let the SAME replay machinery consolidate into it.**

### 7.2 Why naive online learning is the wrong answer (be blunt)

Per-token online SGD on the live model — "train on every message as it arrives" — is the obvious
idea and it is **wrong**: it causes **catastrophic forgetting** (each new gradient overwrites prior
knowledge), it is unstable (one weird input can wreck the model), and it can't be rolled back. Real
brains don't do this either — they *encode fast* (hippocampus, now) and *consolidate slow* (cortex,
during sleep). We copy that.

### 7.3 DESIGN — the LLM consolidation cycle (plugs into the existing sleep loop)

Add **one new offline stage** to the nightly window that already exists (after ingest+weave,
before/around the engine's 3 AM sleep). It never touches the live serving path; it produces new
weights that are hot-swapped in atomically.

1. **What gets replayed.** The *day's lived experience*, sourced from the CLS machinery that's
   already recording it: the traces that **graduated to Universe that day** (the high-value,
   survival-tested memories from `consolidate_into_universe`), the **high-salience episodic events**
   the sleep NREM scan already selects (`sleep.rs` top events), and the **Ryan prompt→reply pairs**
   from `corpus_discord_*.jsonl`. These are exactly the memories biology replays.
2. **How a training batch is formed.** Turn each replayed item into next-token training text:
   prompt→reply pairs become instruction/response examples; graduated Universe facts become short
   completion targets. **Crucially, mix in a reservoir sample of OLD consolidated data** (a fixed
   random slice of prior corpus/weights-era data) so each batch is *new experience + old memory* —
   this is the single most important anti-forgetting lever. Target a **small, bounded batch**
   (e.g. a few hundred–few thousand examples/night), not "everything ever."
3. **LoRA, not full fine-tune (recommended).** Run a **small LoRA update** on the brain module
   (low-rank adapters on the attention/MLP projections), **not** a full-weight fine-tune. Why LoRA
   for the nightly cycle: (a) it fits the 4050 trivially (only adapter params train); (b) it is
   **inherently forgetting-resistant** — the frozen base can't drift; (c) adapters are **tiny,
   versioned, and hot-swappable/rollback-able** — if a night's consolidation degrades eval, discard
   that adapter. Reserve a full fine-tune for rare, larger "deep consolidation" passes if ever needed.
4. **How forgetting is prevented (layered):** (i) **replay old + new** in every batch [primary];
   (ii) **small learning rate + few steps** — nudge, don't overwrite; (iii) **LoRA** caps how far
   weights can move; (iv) optional **EWC / regularization** anchoring important weights toward their
   pretrained values; (v) **eval-gated commit** (§7.4) — a consolidation that regresses held-out
   perplexity is rejected, not shipped.
5. **How the updated weights hot-swap into the live loop.** The generation seam already exists —
   `language_warehouse.rs:474-598` runs `forward → logits → bias → sample → next_token`, and already
   biases logits from lattice evidence (`:584`). The nightly stage writes a new adapter/checkpoint;
   at the next safe idle boundary the engine **atomically loads the new weights** (swap the adapter
   the brain module loads at that seam), flag-gated and reversible. No live-path SGD, ever.

### 7.4 Why this gives the "open part" the owner wants

Fast and slow, exactly like awake-encoding + sleep-consolidation in humans:

- **Fast (immediate, already live):** a new experience is bound into RSHL/hippocampus *now* —
  KAI can recall it within the same conversation (that's `hippocampus.store` at `main.rs:4741`
  + CA3 completion). Nothing to train; it's available instantly.
- **Slow (accrues over nights):** the consolidation cycle replays those experiences and folds the
  durable ones into the LLM brain module's weights, so over weeks KAI's *generation* — not just its
  retrieval — reflects lived experience. The "open part" is this nightly LoRA-on-replay stage: an
  always-available channel for real-time-encoded experience to become consolidated skill over time.

---

## 8. Revised architecture, phases, risks, and decisions (supersedes where noted)

### 8.1 Revised architecture

- **Brain module (trainable LLM).** A small transformer — **recommend ~50–125M params** ("enough
  to start": 125M/GPT-2-small class is the smallest size that generates genuinely coherent text,
  cheap to cloud-pretrain (§6, <~$200), and small enough to LoRA-adapt + run on the 4050). A ~30M
  variant remains the ultra-cheap fallback for Phase 0.
- **RSHL as memory substrate (unchanged, central).** RSHL/hippocampus/Universe stay the fast
  episodic + associative memory and the kNN-retrieval layer around the brain module (§1, Phase 2
  of the original plan). RSHL is *how experience is captured and selected for replay*; the brain
  module is *what consolidates it into fluent generation.*
- **Consolidation loop (new).** The replay-based nightly LoRA stage of §7, wired into the existing
  sleep/hippocampus scheduler — the bridge that turns fast memories into slow weights.

### 8.2 Revised phased path

- **Phase 0 — Local proof-of-concept (UNCHANGED, ~2–4 days, LOCAL).** As in §3: char/4k-BPE,
  `RSHLHead` or a tiny LM, PyTorch loop on the 4050, **overfit one batch to ~0 loss** as the
  green-light gate. No cloud, no consolidation yet. *Verify:* loss→0 on a batch; beats unigram.
- **Phase 1 — CLOUD pretrain of the brain module (NEW, ~2–4 days wall, CLOUD).** *Build:* shard
  the corpus, rent 1× A100 (§6.3), pretrain a 50–125M model in PyTorch, download `.safetensors`.
  *Verify:* held-out perplexity curve down; coherent samples; base loads locally on the 4050.
  *Effort:* low-medium. *Cost:* ~$50–200. Pretrain is paid **once**.
- **Phase 2 — The CONSOLIDATION LOOP (NEW, the core of the idea, ~1–2 weeks, LOCAL).** *Build:*
  the nightly replay→batch→**LoRA** stage of §7.3, sourcing replay from graduated-Universe + sleep
  NREM top events + Discord pairs, mixing in a reservoir of old data; small LR; write a versioned
  adapter. Slot it into the existing ~3–4:30 AM window. *Verify:* after N nights, held-out perplexity
  is flat-or-down (no forgetting) **and** new-experience recall improves; a bad night's adapter is
  rejected by the eval gate. *Effort:* medium. Runs on the 4050.
- **Phase 3 — Hot-swap + eval-over-time (NEW, ~1 week, LOCAL).** *Build:* atomic adapter load at
  the `language_warehouse.rs` seam (flag-gated, reversible); a small **eval-over-time dashboard**
  (perplexity + next-token top-1/5 tracked per consolidation cycle). *Verify:* owner A/Bs KAI with
  the consolidated brain on/off; the time-series shows learning accrue without drift. *Effort:* medium.

### 8.3 Honest risks + mitigations

- **Catastrophic forgetting** (new nights erase old knowledge). *Mitigate:* replay old+new every
  batch; LoRA (frozen base); small LR/few steps; optional EWC; eval-gated commit.
- **Consolidation instability** (a weird day's data destabilizes the model; adapters accumulate
  drift). *Mitigate:* bounded batch size + step count; per-night versioned adapters with rollback;
  periodic "reset to pretrained base + best adapter"; cap how many adapters stack before a
  consolidate-the-adapters pass.
- **Eval drift** (the metric looks fine while quality silently degrades; held-out set goes stale).
  *Mitigate:* a **frozen** held-out set for comparability + a **rotating** fresh set for realism;
  track top-1/5 accuracy alongside perplexity; keep qualitative sample logs; the owner remains the
  human verifier (project rule).
- **Scope creep back into the live engine.** *Mitigate:* all training stays offline/additive like
  `distill_from_bitnet.py`; never set `KAI_NATIVE_BRAIN` from the trainer; hot-swap only at idle,
  flag-gated, reversible.

### 8.4 Open decisions for the owner

1. **Brain-module size to start:** **~30M** (ultra-cheap, weaker fluency) vs **~50–125M**
   (recommended — coherent, still cloud-cheap and 4050-adaptable) vs **~350M** (stretch, low-hundreds $).
2. **Consolidation update type:** **LoRA nightly** (recommended — cheap, safe, rollback-able) vs
   **periodic full fine-tune** (heavier, more capacity, more forgetting risk) vs a **hybrid**
   (nightly LoRA + rare full "deep consolidation").
3. **Cloud provider/instance for Phase-1 pretrain:** **RunPod/Lambda/Vast.ai 1× A100 80 GB**
   (recommended, PyTorch) vs **Google Cloud TPU** (JAX, cheaper-at-scale, more friction) vs
   **GCP A100 VM** (one place for VPS + storage + GPU).
4. **How often consolidation runs:** **nightly** (recommended — matches biology + the existing
   3 AM sleep window) vs **on-idle** (whenever the resource governor says there's headroom) vs
   **manual** (owner triggers a consolidation pass on demand).
5. **Replay source weighting:** how much to weight *Ryan's own pairs* (persona) vs *graduated-
   Universe facts* (knowledge) vs *reservoir of old data* (stability) in each night's batch.

---

### Appendix A — Key file references (verified this pass)
- RSHL representation & ops: `src/core/sparse_vec.rs` (`DIM`/`SPARSITY` :18-22, `encode` :308,
  `cosine` :500, `bundle` :576, `bind` :757, `unbind` :784).
- Adaptive component (Hebbian, not backprop): `src/cognition/corpus_trainer.rs:250-370`.
- Transformer scaffold (Candle, inference-only): `src/cognition/bitnet_llama.rs` (config :36-45).
- Generation loop / integration seam: `src/cognition/language_warehouse.rs:474-598` (sampler
  :909-942).
- Tokenizer: `models/tokenizer.json` (Llama-3 BPE, vocab 128256), `models/BitNet/tokenizer.json`.
- Corpus: `data/harvest.jsonl` (1.39M lines/301MB), `data/training_corpus/bulk_conversations.jsonl`
  (103MB), `data/training_corpus/corpus_discord_*.jsonl` (his pairs), `data/bulk.jsonl`,
  `data/kai-texts.bin`; **`data/transcripts.db` = empty (0 bytes)**.
- Benchmarks (cost, NOT quality): `benchmarks/rshl_vs_llm.mjs` (framing :27-29), `rshl_fullbench.mjs`,
  `rshl_scale_sweep.mjs`.
- Compute deps: `Cargo.toml:64-66` (candle 0.10.2), `:90` (`llm-cuda`); GPU = RTX 4050 Laptop 6 GB.
- Ingestion "training" (not gradient): `overnight_pipeline.py` (teacher/quiz/distill + `PIPELINE_STOP_HOUR`
  :106 stops for engine-side consolidation), `distill_from_bitnet.py`.
- **CLS consolidation machinery (Rev 2, verified):** fast episodic = `src/cognition/hippocampus.rs`
  (`CA3_CAPACITY` :44, `CA1_BUFFER` :47, `complete` :229, `separate` :288, `consolidate_into_universe`
  :367 with gates :395/:405/:419 and `universe.store` graduation :443); one-shot bind on input at
  `main.rs:4741`; driver loop `main.rs:890-929` (every 50 ticks, coherence-gated, telemetry →
  `data/hippocampus_status.json` :914); sleep/replay scheduler `src/cognition/sleep.rs`
  (`should_sleep` :147 ~3:05 AM, `run_cycle` NREM→SWS→REM :177-260, downscale ×0.92 :64).