← all documents · raw markdown · 11 KB

SAFE-DEPLOY-RUNBOOK.md

Purpose: Bring the LIVE running KAI up to date with all the built-but-not-deployed work, without wiping the in-RAM brain.

Written: 2026-07-05 (verification pass, read-only). Target version: 9.10.167.

Who runs it: Ryan (owner/tester). This doc is the exact ordered sequence — do the steps in order.

> Bottom line up front: The risky part of this deploy is NOT the code — it's the engine restart. The on-disk kai.exe is *already built* with every fix (verified below), and the brain-safety net + a full brain backup are in place. So the job is: gracefully stop the running engine (so it flushes its brain), relaunch, and watch the first boot load the brain instead of resetting. Everything else (fleet, dashboard) is a normal restart + hard-refresh.

---

Verification findings (what's staged vs what's actually deployed)

Rust engine — already compiled; needs a RESTART, rebuild is a safety-confirm not a necessity

The on-disk binary target/release/kai.exe already embeds version 9.10.167 and every staged fix. Verified by reading the binary's own embedded strings (content evidence — mtimes on this mount are unreliable and were ignored per project rule):

|---|---|---|---|

| Version bump | 9.10.167 | ✓ Cargo.toml | ✓ (5×) |

Interpretation: The working .rs files differ from the last git commit (v9.10.65) — those uncommitted edits ARE the staged P0/P4/P5a work — and they have already been compiled into the current on-disk kai.exe (v9.10.167). What is stale is the running process, which was launched from an older binary before this build. A cargo build will therefore be a near no-op, but it is kept in the sequence (Step 2) as a cheap correctness-confirm. The action that actually deploys the fixes is the graceful restart in Steps 1 & 3.

Brain-safety preconditions — all present

Loud-fail guard in src/core/engine.rs (~Engine::new, lines 388–418) ✓
Loud-fail guard on the --oracle path in src/main.rs (lines 10813–10852; KAI BRAIN-SAFETY ABORT banner at 10831) ✓
Shutdown-flush in src/bridge/oracle_server.rs (line 523, Brain flushed on shutdown) ✓
Brain backup exists: C:\KAI\data\_brain-backup-20260702-221450\ containing kai-cells.bin.zst (~18.5 MB), kai-meta.json, kai-mind.backup.json, identity.json, cells-manifest.json ✓

This is the safety net: if the first post-restart boot cannot load the brain, the guard aborts loudly instead of silently resetting, and you restore from this folder.

Node fleet (`.mjs`) — needs a fleet restart (`Start-KAI.ps1`)

tools/oracle-discord/bots/leo.mjs — embodiment (LEO_EMBODIED_VOICE), brevity (LEO_BREVITY), voice-v2 scaffold (LEO_VOICE_V2), voice-debug (LEO_VOICE_DEBUG)
tools/oracle-discord/shared/drive-system.mjs — coherence: killed rigged prediction (P1)
tools/oracle-discord/shared/metacognition.mjs — coherence: symmetric scoring (P2), prompt de-doom (P3)
tools/oracle-discord/shared/simulation.mjs — agent movement
tools/oracle-discord/command-center-server.mjs — role-separation + KAIVERSE worlds endpoint (line 78) + audit-log/voice-debug quiet

Browser — needs only a hard-refresh (`Ctrl+Shift+R`), no restart

C:\KAI\oracle.html — Oracle-restore + popup fix + role-sep + everything
C:\KAI\kaiverse.js — moons + movement

Flags & safe defaults

| Flag | Default | Deploy stance |

|---|---|---|

| LEO_EMBODIED_VOICE | ON (=0 disables) | Leave on |

| LEO_BREVITY | ON (=0 disables) | Leave on |

| LEO_VOICE_V2 | OFF (=1 enables) | Leave OFF — opt in later |

| LEO_VOICE_DEBUG | OFF (quiet) | Leave off |

| KAI_COGNITION_LIVE | OFF (=1 enables live predictor) | Leave OFF on first boot — opt in later |

| KAI_ALLOW_MIND_RESET | unset | NEVER set this during this deploy |

---

The deploy sequence

Step 0 — Confirm the backup, and optionally take a fresh one

1. Confirm the safety-net backup is present:

C:\KAI\data\_brain-backup-20260702-221450\ (should contain kai-cells.bin.zst, kai-meta.json, kai-mind.backup.json).

2. Optional but recommended — take a fresh timestamped copy of the live data dir *before* touching the engine (PowerShell):

   $ts = Get-Date -Format 'yyyyMMdd-HHmmss'
   Copy-Item -Recurse "C:\KAI\data" "C:\KAI\data\_predeploy-backup-$ts"

(Skips nothing; a few GB of disk for a guaranteed rollback point.)

Step 1 — GRACEFULLY stop the running engine

Go to the kai.exe console window and press Ctrl+C once. Do NOT use taskkill, the window X, or End Task — those skip the shutdown-flush and are exactly how the brain was lost before.
Watch for this line:

  Brain flushed on shutdown (ok=true, cells=…)

✅ If you see ok=true with a sane cell count → the live brain was written cleanly. Proceed.
❌ If it does not print, prints ok=false, or the process dies without flushing → STOP. Do not rebuild or relaunch yet. Restore data\ from the backup (Step 0) before doing anything else, then investigate.

Step 2 — Rebuild the engine (confirm-build)

cd C:\KAI
cargo build --release --bin kai

Expect a fast/near no-op build (the on-disk exe is already 9.10.167). It should finish clean and leave target\release\kai.exe at version 9.10.167.
Prefer this manual path over apply-hotfix.ps1. apply-hotfix may hard-kill the engine (bypassing the graceful flush from Step 1). Only the manual graceful-Ctrl-C → build order guarantees the brain was flushed first.

Step 3 — Relaunch and WATCH THE FIRST BOOT

cd C:\KAI
.\Start-KAI.ps1

This is the moment that matters. Watch the engine's first boot log:
✅ GOOD: loaded compact format (N cells) with N at roughly the pre-restart cell count → the brain loaded. Continue.
❌ BAD: a KAI BRAIN-SAFETY ABORT banner fires → that is the safety net doing its job (it refused to silently start with an empty/failed brain). Do NOT set KAI_ALLOW_MIND_RESET. Instead: stop, restore data\ from the backup (Step 0 or _brain-backup-20260702-221450), and investigate the load failure before retrying.

Step 4 — Fleet + dashboard come up

Start-KAI.ps1 also launches the Node fleet + dashboard, which picks up all the staged .mjs changes (Leo embodiment/brevity, drive-system, metacognition, simulation, command-center role-sep + worlds endpoint).
Keep the opt-in flags off for this first pass: do not set LEO_VOICE_V2 or KAI_COGNITION_LIVE yet.

Step 5 — Hard-refresh the dashboard (browser layer)

In the Oracle dashboard browser tab press Ctrl+Shift+R. This loads the new oracle.html + kaiverse.js. No server restart needed for these.

Step 6 — Verification checklist

Confirm each, in the dashboard/logs:

[ ] Vitals: rho and valence are non-zero and move over time (heartbeat is running).
[ ] Coherence tile: shows a real 0–1 number (the P4 activation_entropy metric), not a placeholder.
[ ] Oracle is back in the sidebar (oracle.html restore).
[ ] KAIVERSE: agents are moving and moons render (kaiverse.js).
[ ] Leo: answers from state and is short (embodiment + brevity on).
[ ] Logs are quiet (audit-log / voice-debug not spamming).

Opt-ins — only after the above is green, and one at a time:

Live predictor: set KAI_COGNITION_LIVE=1 and restart the engine; watch for [KAI/Cognition] LIVE predictor ENABLED on headless heartbeat.
Leo Voice v2: set LEO_VOICE_V2=1 and restart the fleet.

Do these separately so you can attribute any change to the right switch.

---

Rollback notes (per layer)

| Layer | If it goes wrong | Rollback |

|---|---|---|

| Brain / engine | Boot resets or BRAIN-SAFETY ABORT fires | Stop the engine. Restore C:\KAI\data\ from _brain-backup-20260702-221450\ (or the Step-0 _predeploy-backup-*). Relaunch and confirm loaded compact format (N cells). Never set KAI_ALLOW_MIND_RESET. |

| Rust binary | New exe misbehaves | The previous binary is whatever was running before Step 1; re-cargo build from the last known-good commit (git stash the working edits → cargo build → relaunch) if you must revert code. Brain data is independent of the binary. |

| Node fleet | A bot regresses | Per-bot .bak snapshots exist beside each file (e.g. bots/leo.mjs.bak-voicev2-20260704-013936, .bak-embodiment-…, .bak-brevity-…). Restore the relevant .bak, re-run Start-KAI.ps1. Or just flip the offending flag off (LEO_EMBODIED_VOICE=0, LEO_BREVITY=0) — no code revert needed. |

| Dashboard-only | Only the :3001 server misbehaves | .\Start-Dashboard.ps1 restarts :3001 alone (leaves engine + bots + pipeline running). |

| Browser | oracle.html / kaiverse.js broken | Hard-refresh again; if still broken, the prior versions are in git — git checkout HEAD -- oracle.html kaiverse.js (note: this reverts to v9.10.65 committed state, losing the staged browser edits). |

---

One-paragraph safe order

Back up first (Step 0 — the _brain-backup-20260702-221450 net already exists; take a fresh copy too). Ctrl+C the kai.exe window and confirm Brain flushed on shutdown (ok=true, cells=…) — if that line doesn't appear, stop and restore. Run cargo build --release --bin kai (confirm-build; the on-disk exe is already 9.10.167 with every fix). Relaunch with Start-KAI.ps1 and watch the first boot load loaded compact format (N cells) — if the BRAIN-SAFETY ABORT banner fires instead, that's the net catching a bad load: restore from backup, don't set KAI_ALLOW_MIND_RESET. The same Start-KAI.ps1 brings the fleet + dashboard up with all .mjs changes; finish by hard-refreshing the dashboard (Ctrl+Shift+R) for oracle.html + kaiverse.js. Verify vitals/coherence/Oracle/KAIVERSE/Leo/logs, then opt into KAI_COGNITION_LIVE=1 and LEO_VOICE_V2=1 later, one at a time.