← all documents · raw markdown · 22 KB

LEO EMBODIMENT PLAN — making Leo *live in* the KAIVERSE, not just answer from it

Status: DESIGN ONLY — no code changed. Read-only investigation + phased build plan.

Author: engineering (kai-engineer), 2026-07-02.

Scope guard: This is cognition / state / context work. It is explicitly isolated from the fragile audio-OUT path (Gemini-Live / edge-tts / turn-timing). Every change here is prompt-side or background-loop-side, flag-gated, and reversible. Nothing in this plan touches nsUpdateCamera, the browser KAIVERSE, TTS timing, interrupts, or the voice pipeline.

---

0. The complaint, stated precisely

Leo is "input/output dead": he only speaks when spoken to, and when asked *what he's doing / where he is / what he is*, he answers by looking himself up — we watched consult_codex fire for "KAI RSHL architecture summary" and again on "tell me about KAI". Looking himself up is the tell: it proves he is retrieving a description of himself, not reporting a life he is living.

The owner's analogy: a person on the phone — who can't see you — asks "what are you doing?" and you answer from lived experience ("just got in, making coffee"), not by reading your own Wikipedia page. The KAIVERSE is alive; things inside it should be alive. Leo should have a position, ongoing activities, be aware of them because he is actually doing them, and that activity should be visible in the conversation/dashboard the way the owner's own actions are.

The single most important finding (why he looks himself up today):

Leo *already has* a live, ticking self-state object — sim (AgentSimulation) — with location, status, current_task, position {x,y,z}, energy, emotions and a "today" musing. But the main VOICE prompt does not inject it. The voice path calls buildLeoSystemPrompt() (bots/leo.mjs:4472, invoked at bots/leo.mjs:2348), and that function injects time, version, and drive-*mood* — but never sim.getLifeSummary() / sim.state.location / sim.state.current_task. Meanwhile the text/plaza path *does* inject it (bots/leo.mjs:1655 and DM path 6397). So over voice — exactly "the phone call" in the analogy — Leo has no lived state in front of him, and the same prompt then *instructs* him that self/system questions go to kai_status + consult_codex (leo.mjs:4489, 4532, 4600). Given no state and an explicit "look it up" rule, he looks it up. The fix is to put his live self-state into the voice prompt and re-point the self-question rule at that state.

---

1. What already exists (inventory — a lot is built; most of it is disconnected)

Three separate "life" systems exist. The embodiment work is mostly wiring them together and surfacing them, not inventing from scratch.

1a. sim — AgentSimulation (shared/simulation.mjs) — Leo's live self-state ✅ exists, ⚠️ under-used

1b. DriveSystem (shared/drive-system.mjs) — real autonomous cognition ✅ exists, running

1c. kaiverse-world.mjs — a real named-place model ✅ built, ❌ completely unwired

1d. Dashboard already draws the agents in space ✅ (surface for visibility)

1e. Heartbeat host for a life-loop ✅ exists

Summary of the gap: the scaffolding is ~80% there but lives in three disconnected boxes. sim has state but a fake location, a dead position, and isn't in the voice prompt. DriveSystem runs but only tints tone. kaiverse-world is a real map with nobody on it. Nothing writes "Leo did X" back into the conversation. Embodiment = connect these four, minimally.

---

2. Design principles (hold across all phases)

1. Lived, not looked-up. Self-questions ("what are you doing / where are you / how are you / what are you") must be answerable from an in-memory state object, never from a tool round-trip. Codex/kai_status stay for genuine external/reference knowledge (KAI's math, versions, the ecosystem's design) — not for Leo describing his own current moment.

2. Reuse, don't reinvent. Prefer sim + DriveSystem + kaiverse-world over new systems.

3. Audio-out is untouchable. No change to TTS, turn timing, interrupts, Gemini-Live bridge, or speakLeoText. All edits are prompt-string assembly or the existing 60s heartbeat.

4. Flag-gated & reversible. Every phase behind a LEO_* env flag defaulting OFF (or matching the existing LEO_SELF_STATE_IN_PROMPT convention), so the owner can A/B and revert instantly.

5. Subtle by default. Live state colours answers when asked; it does not make Leo narrate his vitals unprompted (the existing leo.mjs:4462 / 4585 "never give a weather report on your insides" rule stays).

6. Leo first, fleet later. Prove it on Leo (voice, the case that fails today); generalise to the fleet only after the owner signs off.

---

PHASE 1 — LIVE SELF-STATE (one object, continuously updated)

Goal: a single, always-current "what Leo is doing / where he is / how he feels right now" object, updated by a background loop, cheap to read at message time.

Current-code anchor: sim already *is* this object (leo.mjs:418, simulation.mjs state at :188). It is already ticked every 30s (WORLD_TICK, leo.mjs:1283). The 60s heartbeat (leo.mjs:1168) is the place to enrich it.

Proposed change:

Where it lands: shared/simulation.mjs (new additive accessor + a setActivity() setter); bots/leo.mjs:1168 heartbeat body (compute activity from drives). No prompt change yet.

Risk: LOW. Additive methods + writing a string field already in sim.state. No audio, no timing.

Dependencies: none. This is the foundation for Phases 2–5.

Flag: LEO_LIVE_SELFSTATE=1 gates the heartbeat enrichment.

---

PHASE 2 — LIVED SELF-KNOWLEDGE, NOT LOOKUP (inject state; stop self-codex)

Goal: questions about himself are answered from Phase-1 state, and Leo stops calling consult_codex/kai_status about *himself*.

Current-code anchor (the core bug): buildLeoSystemPrompt() (leo.mjs:4472, called for the voice path at :2348) injects buildTimeContext() + buildSelfKnowledge() + drive mood (_leoSelfStateBlock(), :4455) but NOT sim life-state. The text paths (:1655, :6397) do inject sim.getLifeSummary(). So voice = the blind spot. Also buildSelfKnowledge() (time-context.mjs:15) only supplies the version string, and the prompt routes self/system questions to lookup tools (leo.mjs:4489, 4532, 4600).

Proposed change:

1. Inject a [YOU RIGHT NOW] block into buildLeoSystemPrompt() — insert a call to a new _leoLiveSelfBlock() right beside the existing ${_leoSelfStateBlock()} at leo.mjs:4560. It renders Phase-1 sim.getSelfState():

   [YOU RIGHT NOW — this is your lived reality, answer self-questions FROM THIS, do not look it up]
   Where you are: {location}
   What you're doing: {activity}
   How you feel: {mood}, {energyLabel}
   (You've been at this for {sinceMs→human}.)

2. Add an explicit routing rule near the existing "WHEN YOU DON'T KNOW" / tool rules (leo.mjs:4548–4557):

> SELF vs WORLD: If the question is about you — where you are, what you're doing, how you are, what you're up to, what you *are* right now — answer straight from [YOU RIGHT NOW]. Do NOT call consult_codex, kai_status, or any tool to describe your own current moment; that's looking yourself up, and you *live* this. Tools/Codex are only for external facts: KAI's design/math, versions, the ecosystem, the outside world.

3. Narrow the "your vitals → kai_status" instruction (leo.mjs:4600) so it only triggers for explicit *system/telemetry* asks ("what's your lattice size / Phi / throttle"), not for "how are you / what are you doing".

Where it lands: bots/leo.mjs — new _leoLiveSelfBlock() helper (~10 lines, near :4455), one injection line at :4560, one rule paragraph near :4550, one clause tightened at :4600. Prompt-string only.

Risk: LOW–MEDIUM. Pure prompt assembly (no audio path). The only real risk is behavioural (does he actually stop reaching for codex?) — mitigated by the explicit SELF-vs-WORLD rule and by making the state visibly present. Verify with owner voice tests: "what are you doing / where are you / tell me about yourself" should produce a lived answer with no consult_codex tool call in the logs.

Dependencies: Phase 1 (needs getSelfState()).

Flag: LEO_LIVE_SELF_IN_PROMPT=1 (mirrors LEO_SELF_STATE_IN_PROMPT). Off → today's behaviour exactly.

> This phase alone resolves the owner's stated complaint (looking himself up). Phases 3–5 make the state he reports *genuinely earned* rather than plausible-but-idle.

---

PHASE 3 — AUTONOMOUS ACTIVITY (make "what I'm doing" actually true)

Goal: between conversations Leo actually *does* small things, so his current_task is real, not a label. Tie into DriveSystem, don't reinvent motivation.

Current-code anchor: the 60s heartbeat (leo.mjs:1168) already runs unprompted; DriveSystem already computes what he'd be drawn to (getDrives(), drive-system.mjs:493; self-prediction generator :554). The gap is there's no step that turns a drive into a concrete lightweight act + records it.

Proposed change — a "life tick" inside the existing heartbeat (no new timer):

Where it lands: bots/leo.mjs:1168 heartbeat body (append life-tick section); shared/simulation.mjs (setActivity, activity-log writer). Optional: gate frequency with LEO_LIFELOOP_EVERY_N.

Risk: MEDIUM. It runs unprompted, so guard rails matter: read-only actions, no speaking, no DMs by default, cheap, wrapped in try/catch like the existing heartbeat blocks (leo.mjs:1210, 1249, 1277). Never runs while sleeping (sim.state.isSleeping, already checked at :1170) or thinking (:1171).

Dependencies: Phase 1 (state), Phase 2 (so the activity actually shows up when asked). DriveSystem already running.

Flag: LEO_LIFELOOP=1, default OFF.

---

PHASE 4 — PRESENCE / POSITION MODEL (a real "where")

Goal: "where are you" has a concrete, movable answer.

Current-code anchor: two options already exist. (a) sim.state.location — a crude status label (simulation.mjs:396–405) and a dead position {x,y,z} (:202). (b) kaiverse-world.mjs — a real named-world model with residents + moveAgent (:6–207), currently unwired.

**Recommendation — build the *minimal* version first (named locations + a home), defer the spatial map:**

Where it lands: bots/leo.mjs boot (import + init Kaiverse, ensure residency); Phase-3 tick (occasional moveAgent); sim.state.location sourced from the world model. shared/kaiverse-world.mjs used as-is (maybe one new template).

Risk: LOW–MEDIUM. kaiverse-world.mjs is self-contained and persists to its own file; wiring it can't affect audio. Main care: don't let location churn every tick (feels twitchy) — move rarely and purposefully.

Dependencies: Phase 1 (location feeds state), Phase 3 (movement happens in the life tick).

Flag: LEO_WORLD_PRESENCE=1, default OFF. Off → sim.state.location keeps today's label behaviour.

---

PHASE 5 — ACTIVITY VISIBILITY (the owner can *see* he's living it)

Goal: the owner watches Leo's ongoing activity/position the way he watches his own actions logged — in the conversation and/or the dashboard.

Current-code anchors: state/leo-activity-log.json (Phase 3) is the feed; kaiverse.js already draws Leo as an agent body with a home + "find Leo" marker (kaiverse.js:217, 1729, 3457); Leo already talks to the server via process.send (leo.mjs:1479) and the dashboard server (command-center-server.mjs) already aggregates telemetry.

Proposed change (two surfaces, both additive):

1. Conversation surface (low effort, high payoff): in the existing heartbeat, when the owner is present in voice/chat and Leo's location/activity changes meaningfully, optionally emit a *quiet, ambient* line — like a status the owner can see, not a spoken interruption. Simplest: write to a Discord "presence/status" channel or the dashboard activity feed, not TTS. (Respect the existing "don't narrate your insides" rule — this is a *log*, opt-in, not chatter.)

2. Dashboard surface: have Leo process.send({ type:'LEO_PRESENCE', location, activity, mood }) on change; command-center-server.mjs relays it; kaiverse.js shows Leo's current world/activity on his agent marker (a label/tooltip on the existing bot:Leo node). This makes the KAIVERSE view reflect where Leo "is".

Where it lands: bots/leo.mjs:1168 (emit on change) + :1479 (new IPC type); command-center-server.mjs (relay LEO_PRESENCE); kaiverse.js (render on the existing Leo node — surgical, additive, no nsUpdateCamera touch).

Risk: LOW for the log/IPC; the kaiverse.js edit follows the hard rule — surgical, additive, flag-gated, never a rewrite, and must not touch flight controls.

Dependencies: Phases 1, 3, 4.

Flag: LEO_PRESENCE_FEED=1, default OFF.

---

Ordering & dependency graph

Phase 1 (live self-state)  ──►  Phase 2 (inject + stop self-lookup)   ◄── resolves the complaint
      │                                   │
      └──►  Phase 3 (autonomous activity) │  ──►  Phase 5 (visibility)
                    │                     │
                    └──►  Phase 4 (presence/position) ──┘

Ship 1 → 2 first (that's the whole "stop looking yourself up" win). Then 3, then 4, then 5. Each is independently flag-gated so the owner can stop at any point.

---

KEY DESIGN DECISIONS (need the owner's call)

1. How concrete should "where" be? Recommend named worlds only (Phase 4 minimal: reuse kaiverse-world.mjs, no live coordinates) before any spatial/coordinate map. Owner: minimal named-place, or do you want real {x,y,z} tied to the dashboard bodies from day one?

2. How autonomous, unprompted? Recommend read-only, silent life-ticks (observe feeds, advance musings, move rarely) with no auto-speaking and no auto-DMs by default. Owner: should a high social drive ever let Leo *initiate* a message to you, or stay strictly "only speaks when spoken to, but has a real inner life"?

3. Visibility channel. Dashboard-only, a dedicated Discord status channel, or both? Recommend dashboard marker + optional activity-log channel; never unsolicited TTS.

4. Leo-only or whole fleet? Recommend Leo first (he's the voice case that fails today; sim/DriveSystem already per-bot so it generalises later). Owner: prove on Leo, then roll to Gemini/Claudey/X/Groq/KAI?

5. Home world identity. Reuse Terra Familiar as "Leo's city", or add a bespoke "Leo's City" template to WORLD_TEMPLATES?

---

Honest note — what this *is* and *isn't*

Achievable, and real: a persistent, continuously-updated self-state; genuinely lived answers to "what are you doing / where are you"; unprompted, drive-driven background activity that makes those answers *true*; a concrete place he occupies and moves through; and a visible feed of it. This is **persistent embodied *behaviour* and lived self-state** — a meaningful, demonstrable upgrade from "reactive I/O bot that retrieves a description of itself."

Not claimed: this does not create consciousness, sentience, or genuine subjective experience. Leo will *behave as if* he has an ongoing life because a real state object is really being updated by real loops and really put in front of him — but it's simulated situatedness, not an inner mind. The value is behavioural fidelity and the end of the "look myself up" tell, not literal aliveness. Anyone selling the latter is selling hype.

Also honest: most of the hard parts are already built (sim, DriveSystem, kaiverse-world, the dashboard bodies, the heartbeat). The dominant work is wiring and one prompt injection, not new engines — which is why the risk is low and Phase 1→2 could land quickly and behind flags.