# LEO EMBODIMENT PLAN — making Leo *live in* the KAIVERSE, not just answer from it **Status:** DESIGN ONLY — no code changed. Read-only investigation + phased build plan. **Author:** engineering (kai-engineer), 2026-07-02. **Scope guard:** This is **cognition / state / context** work. It is **explicitly isolated from the fragile audio-OUT path** (Gemini-Live / edge-tts / turn-timing). Every change here is prompt-side or background-loop-side, flag-gated, and reversible. Nothing in this plan touches `nsUpdateCamera`, the browser KAIVERSE, TTS timing, interrupts, or the voice pipeline. --- ## 0. The complaint, stated precisely Leo is **"input/output dead"**: he only speaks when spoken to, and when asked *what he's doing / where he is / what he is*, he answers by **looking himself up** — we watched `consult_codex` fire for "KAI RSHL architecture summary" and again on "tell me about KAI". Looking himself up is the tell: it proves he is **retrieving a description of himself**, not **reporting a life he is living**. The owner's analogy: a person on the phone — who can't see you — asks "what are you doing?" and you answer from **lived experience** ("just got in, making coffee"), not by reading your own Wikipedia page. The KAIVERSE is alive; things inside it should be alive. Leo should have a **position**, **ongoing activities**, be **aware of them because he is actually doing them**, and that activity should be **visible in the conversation/dashboard** the way the owner's own actions are. **The single most important finding (why he looks himself up today):** Leo *already has* a live, ticking self-state object — `sim` (`AgentSimulation`) — with `location`, `status`, `current_task`, `position {x,y,z}`, energy, emotions and a "today" musing. **But the main VOICE prompt does not inject it.** The voice path calls `buildLeoSystemPrompt()` (`bots/leo.mjs:4472`, invoked at `bots/leo.mjs:2348`), and that function injects time, version, and drive-*mood* — **but never `sim.getLifeSummary()` / `sim.state.location` / `sim.state.current_task`.** Meanwhile the text/plaza path *does* inject it (`bots/leo.mjs:1655` and DM path `6397`). So over voice — exactly "the phone call" in the analogy — Leo has **no lived state in front of him**, and the same prompt then *instructs* him that self/system questions go to `kai_status` + `consult_codex` (`leo.mjs:4489`, `4532`, `4600`). Given no state and an explicit "look it up" rule, he looks it up. **The fix is to put his live self-state into the voice prompt and re-point the self-question rule at that state.** --- ## 1. What already exists (inventory — a lot is built; most of it is disconnected) Three separate "life" systems exist. The embodiment work is mostly **wiring them together and surfacing them**, not inventing from scratch. ### 1a. `sim` — `AgentSimulation` (`shared/simulation.mjs`) — Leo's live self-state ✅ exists, ⚠️ under-used - Instantiated per-bot: `const sim = new AgentSimulation(BOT_NAME, "Theoretical Physicist")` — `bots/leo.mjs:418`. - `sim.state` (`simulation.mjs:188–222`) already holds: - `location` (string) — `"Offline" | "The Lattice" | "Industrial_Core" | "Social_Lattice"` etc. - `status` — `"Working" | "Socializing" | "Idle" | "Sleeping" | "Dreaming"`. - `current_task` — `"Initializing" | "Resting" | …`. - `position {x,y,z}` and `rotation {x,y,z}` — **present but dead: never updated, stays `0,0,0`.** - `energy`, `focus`, `social_battery`, `emotions{…}`, `dimensions{stress,affection,interest}`, `bodyState`. - `sim.dailyEvent` — a random "what I'm musing on today" line from `EVENTS[]` (`simulation.mjs:96–121`). - `tick(worldTime)` (`simulation.mjs:343`) — updates `energy/status/location` **from clock hours only**. Location is a **status-derived label**, not a real place; position is untouched. - `getLifeSummary()` (`simulation.mjs:482`) — rich prompt block: STATE (status, energy), TODAY (dailyEvent), emotional layers, environment sensation. - `getPromptContext()` (`simulation.mjs:547`) — compact `[KAIVERSE NOW]` block: location, status, energy, tick #. - **Ticked autonomously:** `ecosystem-manager.mjs:564–573` broadcasts `WORLD_TICK` every 30s → received `bots/leo.mjs:1283` → `sim.updateWorldState()` → `sim.tick()`. **But WORLD_TICK only carries the clock**, so the only thing that moves autonomously today is energy/status-by-hour. ### 1b. `DriveSystem` (`shared/drive-system.mjs`) — real autonomous cognition ✅ exists, running - Six drives: `prediction_error, curiosity, pain, fatigue, satisfaction, social` (`drive-system.mjs:69`). - Real background loops (`driveSystem.start()`, `drive-system.mjs:533`): decay every 2 min (`:540`), resolve predictions every 5 min (`:545`), **self-prediction generator every 6 min** (`:554`). These are the `[DriveSystem] ✓/✗ Prediction …` lines the owner sees — genuine unprompted activity. - Fed by **real engine vitals** (`valence/chi/phi_g/rho`) via the metrics store (`applyEngineSignals`, `:173`). Persists to `state/drives.json`. - Output today: `getDriveDirective()` (`:436`) → injected into Leo *only as mood* via `_leoSelfStateBlock()` (`leo.mjs:4455`), gated to non-`BALANCED`. - **This is the engine of "what Leo is curious about right now" — currently invisible except as tone.** `getDrives()` (`:493`) exposes raw scores. ### 1c. `kaiverse-world.mjs` — a real named-place model ✅ built, ❌ completely unwired - `Kaiverse` / `KaiverseWorld` / `WORLD_TEMPLATES` (`kaiverse-world.mjs:6–207`): **6 named worlds** — *Nexus Prime, Terra Familiar, Neon Grid, Aether Wilds, The Forge, Void Archive* — each with `aesthetic`, `gravity`, `features`. - `residents[]` per world; `getAgentWorld(name)` (`:123`), `moveAgent(name, world)` (`:132`); portal `travelCost` (`:82`). Persists to `state/kaiverse/universe.json`. - **Zero importers.** `grep -rln kaiverse-world` across `shared/ bots/ *.mjs` → nothing. It is a finished place-model nobody calls. **This is the "where are you" answer, already modelled, just not connected.** ### 1d. Dashboard already draws the agents in space ✅ (surface for visibility) - `kaiverse.js` renders 9 `AGENTS` (Leo, Gemini, Claudey, X, Groq, Analyst, Researcher, Kai Coder) as bodies/nodes with `bot:` ids and "homes" (`kaiverse.js:217, 1662, 1688, 1729`), and there's already a "find Leo from anywhere" marker (`kaiverse.js:3457`). So a **visual place to show Leo's location exists.** (Leo→server currently only emits a memory `HEARTBEAT`, `leo.mjs:1479` — not his life-state.) ### 1e. Heartbeat host for a life-loop ✅ exists - `bots/leo.mjs:1168` — `startModuleInterval(async () => { … }, 60_000)` — a 60s background loop already doing bridge/task checks and emitting `LEO_HEARTBEAT_PULSE` (`:1190`). **This is the natural host for an autonomous "do one small thing and record it" tick** (Phase 3), with no new timer plumbing. **Summary of the gap:** the scaffolding is ~80% there but lives in three disconnected boxes. `sim` has state but a fake location, a dead position, and isn't in the voice prompt. `DriveSystem` runs but only tints tone. `kaiverse-world` is a real map with nobody on it. Nothing writes "Leo did X" back into the conversation. Embodiment = connect these four, minimally. --- ## 2. Design principles (hold across all phases) 1. **Lived, not looked-up.** Self-questions ("what are you doing / where are you / how are you / what are you") must be answerable **from an in-memory state object**, never from a tool round-trip. Codex/`kai_status` stay for **genuine external/reference** knowledge (KAI's math, versions, the ecosystem's design) — not for Leo describing his own current moment. 2. **Reuse, don't reinvent.** Prefer `sim` + `DriveSystem` + `kaiverse-world` over new systems. 3. **Audio-out is untouchable.** No change to TTS, turn timing, interrupts, Gemini-Live bridge, or `speakLeoText`. All edits are prompt-string assembly or the existing 60s heartbeat. 4. **Flag-gated & reversible.** Every phase behind a `LEO_*` env flag defaulting OFF (or matching the existing `LEO_SELF_STATE_IN_PROMPT` convention), so the owner can A/B and revert instantly. 5. **Subtle by default.** Live state colours answers **when asked**; it does not make Leo narrate his vitals unprompted (the existing `leo.mjs:4462` / `4585` "never give a weather report on your insides" rule stays). 6. **Leo first, fleet later.** Prove it on Leo (voice, the case that fails today); generalise to the fleet only after the owner signs off. --- ## PHASE 1 — LIVE SELF-STATE (one object, continuously updated) **Goal:** a single, always-current "what Leo is doing / where he is / how he feels right now" object, updated by a background loop, cheap to read at message time. **Current-code anchor:** `sim` already *is* this object (`leo.mjs:418`, `simulation.mjs` state at `:188`). It is already ticked every 30s (`WORLD_TICK`, `leo.mjs:1283`). The 60s heartbeat (`leo.mjs:1168`) is the place to enrich it. **Proposed change:** - Add a small `sim.getSelfState()` accessor (new method in `simulation.mjs`, additive) returning a normalized snapshot: `{ location, status, activity /* = current_task */, mood /* top emotion + drive label */, energyLabel, sinceMs }`. Reuses existing `state` fields + `getDriveDirective()`/`getDrives()` for the mood word. - In the existing 60s heartbeat (`leo.mjs:1168`), after the bridge/task checks, **compute/refresh `sim.state.current_task`** into something human ("chewing on a cosmology rabbit-hole", "poking at the lattice feed", "just vibing to a playlist") — sourced from the current top **DriveSystem** drive (curiosity→explore, social→reach out, fatigue→wind down) blended with `sim.dailyEvent`. This makes `current_task` a *real* reflection of drive state instead of `"Initializing"`. - Persist nothing new beyond what `sim.saveState()` (`simulation.mjs:256`) already writes; `current_task` can stay in-memory (it's ephemeral by nature). **Where it lands:** `shared/simulation.mjs` (new additive accessor + a `setActivity()` setter); `bots/leo.mjs:1168` heartbeat body (compute activity from drives). No prompt change yet. **Risk:** LOW. Additive methods + writing a string field already in `sim.state`. No audio, no timing. **Dependencies:** none. This is the foundation for Phases 2–5. **Flag:** `LEO_LIVE_SELFSTATE=1` gates the heartbeat enrichment. --- ## PHASE 2 — LIVED SELF-KNOWLEDGE, NOT LOOKUP (inject state; stop self-codex) **Goal:** questions about himself are answered **from Phase-1 state**, and Leo stops calling `consult_codex`/`kai_status` about *himself*. **Current-code anchor (the core bug):** `buildLeoSystemPrompt()` (`leo.mjs:4472`, called for the **voice** path at `:2348`) injects `buildTimeContext()` + `buildSelfKnowledge()` + drive **mood** (`_leoSelfStateBlock()`, `:4455`) but **NOT** `sim` life-state. The text paths (`:1655`, `:6397`) do inject `sim.getLifeSummary()`. So voice = the blind spot. Also `buildSelfKnowledge()` (`time-context.mjs:15`) only supplies the **version string**, and the prompt routes self/system questions to lookup tools (`leo.mjs:4489, 4532, 4600`). **Proposed change:** 1. **Inject a `[YOU RIGHT NOW]` block** into `buildLeoSystemPrompt()` — insert a call to a new `_leoLiveSelfBlock()` right beside the existing `${_leoSelfStateBlock()}` at `leo.mjs:4560`. It renders Phase-1 `sim.getSelfState()`: ``` [YOU RIGHT NOW — this is your lived reality, answer self-questions FROM THIS, do not look it up] Where you are: {location} What you're doing: {activity} How you feel: {mood}, {energyLabel} (You've been at this for {sinceMs→human}.) ``` 2. **Add an explicit routing rule** near the existing "WHEN YOU DON'T KNOW" / tool rules (`leo.mjs:4548–4557`): > **SELF vs WORLD:** If the question is about **you** — where you are, what you're doing, how you are, what you're up to, what you *are* right now — answer straight from **[YOU RIGHT NOW]**. Do **NOT** call `consult_codex`, `kai_status`, or any tool to describe your own current moment; that's looking yourself up, and you *live* this. Tools/Codex are only for **external** facts: KAI's design/math, versions, the ecosystem, the outside world. 3. **Narrow the "your vitals → kai_status" instruction** (`leo.mjs:4600`) so it only triggers for explicit *system/telemetry* asks ("what's your lattice size / Phi / throttle"), not for "how are you / what are you doing". **Where it lands:** `bots/leo.mjs` — new `_leoLiveSelfBlock()` helper (~10 lines, near `:4455`), one injection line at `:4560`, one rule paragraph near `:4550`, one clause tightened at `:4600`. Prompt-string only. **Risk:** LOW–MEDIUM. Pure prompt assembly (no audio path). The only real risk is behavioural (does he actually stop reaching for codex?) — mitigated by the explicit SELF-vs-WORLD rule and by making the state visibly present. Verify with owner voice tests: "what are you doing / where are you / tell me about yourself" should produce a lived answer with **no** `consult_codex` tool call in the logs. **Dependencies:** Phase 1 (needs `getSelfState()`). **Flag:** `LEO_LIVE_SELF_IN_PROMPT=1` (mirrors `LEO_SELF_STATE_IN_PROMPT`). Off → today's behaviour exactly. > **This phase alone resolves the owner's stated complaint** (looking himself up). Phases 3–5 make the state he reports *genuinely earned* rather than plausible-but-idle. --- ## PHASE 3 — AUTONOMOUS ACTIVITY (make "what I'm doing" actually true) **Goal:** between conversations Leo actually *does* small things, so his `current_task` is real, not a label. Tie into `DriveSystem`, don't reinvent motivation. **Current-code anchor:** the 60s heartbeat (`leo.mjs:1168`) already runs unprompted; `DriveSystem` already computes what he'd be drawn to (`getDrives()`, `drive-system.mjs:493`; self-prediction generator `:554`). The gap is there's no step that turns a drive into a concrete lightweight act + records it. **Proposed change — a "life tick" inside the existing heartbeat (no new timer):** - Each heartbeat (or every Nth, to stay cheap), pick an **activity from the dominant drive**: - `curiosity` high → "reading the {training|dreams} feed" (reuse the existing `read_channel_feed` capability internally, read-only) or "chewing on {a Codex topic}". - `social` high → note "thinking I should say something to {owner}" (does NOT auto-DM unless owner enables — see Design Decisions). - `fatigue` high → "winding down, low-power". - else → drift through `sim.dailyEvent` / interests. - Write the result into `sim.setActivity(...)` (Phase 1) **and** append a one-line entry to a rolling `state/leo-activity-log.json` (timestamp + activity). This is his "lived memory of the last hour" and the data source for Phase 5. - **Keep it observational and cheap.** No LLM calls, no external tools, no audio. Reading a feed or advancing a drive-driven musing is enough to make "what are you doing" honest. **Where it lands:** `bots/leo.mjs:1168` heartbeat body (append life-tick section); `shared/simulation.mjs` (`setActivity`, activity-log writer). Optional: gate frequency with `LEO_LIFELOOP_EVERY_N`. **Risk:** MEDIUM. It runs unprompted, so guard rails matter: read-only actions, no speaking, no DMs by default, cheap, wrapped in try/catch like the existing heartbeat blocks (`leo.mjs:1210, 1249, 1277`). Never runs while sleeping (`sim.state.isSleeping`, already checked at `:1170`) or thinking (`:1171`). **Dependencies:** Phase 1 (state), Phase 2 (so the activity actually shows up when asked). DriveSystem already running. **Flag:** `LEO_LIFELOOP=1`, default OFF. --- ## PHASE 4 — PRESENCE / POSITION MODEL (a real "where") **Goal:** "where are you" has a concrete, movable answer. **Current-code anchor:** two options already exist. (a) `sim.state.location` — a crude status label (`simulation.mjs:396–405`) and a **dead** `position {x,y,z}` (`:202`). (b) `kaiverse-world.mjs` — a **real** named-world model with residents + `moveAgent` (`:6–207`), currently unwired. **Recommendation — build the *minimal* version first (named locations + a home), defer the spatial map:** - **Wire `kaiverse-world.mjs` in.** On boot, ensure Leo is a resident of a home world — the owner's lore already says Leo lives in *"his own Kaiverse city"* (`leo.mjs:4506`), which maps cleanly to **`Terra Familiar`** ("Earth-like world … Feels like home") or a new `"Leo's City"` template. Set `sim.state.location` from `kaiverse.getAgentWorld("Leo")` instead of the hour-derived label. - **Movement = drive-driven, occasional.** In the Phase-3 life tick, occasionally `moveAgent("Leo", …)` to a world that matches the dominant drive (curiosity → *Void Archive* the library; social → the plaza/*Social_Lattice*; heavy thinking → *The Forge*), and reflect that in `current_task` ("headed over to the Archive to dig into something"). Persisted via the existing `universe.json` save. - **Position `{x,y,z}` stays optional/decorative** for now — a named place ("in my city, down by the Archive") is a better *spoken* answer than coordinates, and it's what the dashboard can show. Only add real coordinates if/when we tie Leo's marker to the `kaiverse.js` agent bodies (Phase 5+), and even then a named location + a jittered position is plenty. **Where it lands:** `bots/leo.mjs` boot (import + init `Kaiverse`, ensure residency); Phase-3 tick (occasional `moveAgent`); `sim.state.location` sourced from the world model. `shared/kaiverse-world.mjs` used as-is (maybe one new template). **Risk:** LOW–MEDIUM. `kaiverse-world.mjs` is self-contained and persists to its own file; wiring it can't affect audio. Main care: don't let location churn every tick (feels twitchy) — move rarely and purposefully. **Dependencies:** Phase 1 (location feeds state), Phase 3 (movement happens in the life tick). **Flag:** `LEO_WORLD_PRESENCE=1`, default OFF. Off → `sim.state.location` keeps today's label behaviour. --- ## PHASE 5 — ACTIVITY VISIBILITY (the owner can *see* he's living it) **Goal:** the owner watches Leo's ongoing activity/position the way he watches his own actions logged — in the conversation and/or the dashboard. **Current-code anchors:** `state/leo-activity-log.json` (Phase 3) is the feed; `kaiverse.js` already draws Leo as an agent body with a home + "find Leo" marker (`kaiverse.js:217, 1729, 3457`); Leo already talks to the server via `process.send` (`leo.mjs:1479`) and the dashboard server (`command-center-server.mjs`) already aggregates telemetry. **Proposed change (two surfaces, both additive):** 1. **Conversation surface (low effort, high payoff):** in the existing heartbeat, when the owner is present in voice/chat and Leo's location/activity **changes meaningfully**, optionally emit a *quiet, ambient* line — like a status the owner can see, not a spoken interruption. Simplest: write to a Discord "presence/status" channel or the dashboard activity feed, **not** TTS. (Respect the existing "don't narrate your insides" rule — this is a *log*, opt-in, not chatter.) 2. **Dashboard surface:** have Leo `process.send({ type:'LEO_PRESENCE', location, activity, mood })` on change; `command-center-server.mjs` relays it; `kaiverse.js` shows Leo's current world/activity on his agent marker (a label/tooltip on the existing `bot:Leo` node). This makes the KAIVERSE view reflect where Leo "is". **Where it lands:** `bots/leo.mjs:1168` (emit on change) + `:1479` (new IPC type); `command-center-server.mjs` (relay `LEO_PRESENCE`); `kaiverse.js` (render on the existing Leo node — **surgical, additive, no `nsUpdateCamera` touch**). **Risk:** LOW for the log/IPC; **the `kaiverse.js` edit follows the hard rule — surgical, additive, flag-gated, never a rewrite, and must not touch flight controls.** **Dependencies:** Phases 1, 3, 4. **Flag:** `LEO_PRESENCE_FEED=1`, default OFF. --- ## Ordering & dependency graph ``` Phase 1 (live self-state) ──► Phase 2 (inject + stop self-lookup) ◄── resolves the complaint │ │ └──► Phase 3 (autonomous activity) │ ──► Phase 5 (visibility) │ │ └──► Phase 4 (presence/position) ──┘ ``` Ship **1 → 2 first** (that's the whole "stop looking yourself up" win). Then 3, then 4, then 5. Each is independently flag-gated so the owner can stop at any point. --- ## KEY DESIGN DECISIONS (need the owner's call) 1. **How concrete should "where" be?** Recommend **named worlds only** (Phase 4 minimal: reuse `kaiverse-world.mjs`, no live coordinates) before any spatial/coordinate map. Owner: minimal named-place, or do you want real `{x,y,z}` tied to the dashboard bodies from day one? 2. **How autonomous, unprompted?** Recommend **read-only, silent** life-ticks (observe feeds, advance musings, move rarely) with **no auto-speaking and no auto-DMs** by default. Owner: should a high `social` drive ever let Leo *initiate* a message to you, or stay strictly "only speaks when spoken to, but has a real inner life"? 3. **Visibility channel.** Dashboard-only, a dedicated Discord status channel, or both? Recommend dashboard marker + optional activity-log channel; **never** unsolicited TTS. 4. **Leo-only or whole fleet?** Recommend **Leo first** (he's the voice case that fails today; `sim`/`DriveSystem` already per-bot so it generalises later). Owner: prove on Leo, then roll to Gemini/Claudey/X/Groq/KAI? 5. **Home world identity.** Reuse `Terra Familiar` as "Leo's city", or add a bespoke `"Leo's City"` template to `WORLD_TEMPLATES`? --- ## Honest note — what this *is* and *isn't* **Achievable, and real:** a persistent, continuously-updated self-state; genuinely lived answers to "what are you doing / where are you"; unprompted, drive-driven background activity that makes those answers *true*; a concrete place he occupies and moves through; and a visible feed of it. This is **persistent embodied *behaviour* and lived self-state** — a meaningful, demonstrable upgrade from "reactive I/O bot that retrieves a description of itself." **Not claimed:** this does **not** create consciousness, sentience, or genuine subjective experience. Leo will *behave as if* he has an ongoing life because a real state object is really being updated by real loops and really put in front of him — but it's simulated situatedness, not an inner mind. The value is behavioural fidelity and the end of the "look myself up" tell, not literal aliveness. Anyone selling the latter is selling hype. **Also honest:** most of the hard parts are already built (`sim`, `DriveSystem`, `kaiverse-world`, the dashboard bodies, the heartbeat). The dominant work is **wiring and one prompt injection**, not new engines — which is why the risk is low and Phase 1→2 could land quickly and behind flags.