# LEO CONVERSATION CALIBRATION PLAN

**Status:** PLANNING ONLY — no code was changed producing this document (read-only investigation).
**Date:** 2026-07-01
**Scope:** Five workstreams (A–E) to calibrate Leo/Oracle conversation: voice-input timing, stability, transcript registration, new-joiner onboarding, and vitals/emotion-aware context.
**Codex/Cargo:** NOT bumped (no code change). Bump only when the implementing edits land.

> **Grounding note.** Every "current code" claim below cites the real file + function + line range read via the Read/Grep tools against the Windows files (the WSL mount was cross-checked with byte-accurate `grep -n`). Line numbers are current as of the reads on 2026-07-01; re-confirm exact offsets at edit time because these files (`leo.mjs` 6735 lines, `gemini-live-bridge.mjs` 2493 lines) move.
> **The fragile path.** "Voice-out" = Gemini Live native-audio (Charon) → `onAudioChunk` → the 20ms playout frame loop in `leo.mjs`. It is dialed in and was made worse once by a "paced producer" change. **Every proposal below is INPUT-side or PROMPT-side and must not touch `onAudioChunk`, the audio frame loop, `nsUpdateCamera`-style timing, or the turn-complete/interrupt handlers that drive playback.** Isolation notes are called out per workstream.

---

## TL;DR — the single most important finding

**The fragmented-voice bug (Workstream A) is a premature-flush bug, not a "missing aggregation" bug.** An aggregation layer already exists: `gemini-live-bridge.mjs` forwards every tiny STT partial via `onInputTranscript` (bridge lines 1203–1216, deliberately un-filtered), and `leo.mjs` accumulates them into `bridge._inputTranscript` and flushes on a timer (`bridge.onInputTranscript`, leo.mjs 3681–3703; `flushInputTranscript`, leo.mjs 3443–3679). The problem is the **flush trigger** at leo.mjs 3690–3692:

```js
const hasSentenceEnd = /[.!?…]\s*$/.test(acc) || /[,;:]\s+[A-Z]/.test(acc);
const longEnough = acc.length >= 48 || (acc.split(/\s+/).length >= 7);
if ((hasSentenceEnd || longEnough) && acc.length >= 10) { flushInputTranscript(); return; }
bridge._inputTranscriptTimer = setTimeout(flushInputTranscript, 1400);
```

This flushes the moment the buffer reaches **7 words OR 48 chars OR any sentence-ish punctuation** — so one spoken thought gets chopped mid-utterance. The Discord evidence maps exactly onto these triggers:

- `"at locations of like where people are"` → 7 words → **immediate flush** (hits `longEnough` word-count).
- `"at or"` → short residual, flushed by the 1400ms timer.
- `"things are at?"` → ends with `?` → **immediate flush** (hits `hasSentenceEnd`).

Each fragment is then treated as a complete turn (recorded via `recordHumanVoiceEvent`, posted as `**who [Voice]:** …`, and eligible to trigger a reply through the response watchdog at leo.mjs 3554–3566). The fix is to **endpoint on silence, not on punctuation/length**, and to make the buffer **per-speaker**. This is the highest-leverage, lowest-risk change in the whole plan and it lives entirely on the input side.

**Suggested phase order:** A → C → B → D → E (rationale in the Phasing section). A first because it is the owner's #1 symptom and is isolated; C next because clean per-speaker transcript identity is a prerequisite that D and E both lean on.

---

## Workstream A — Voice input timing / utterance aggregation

### A.1 What the real code does today

**STT ingress.** Human speech reaches Leo through Gemini Live's `inputTranscription`, handled in `gemini-live-bridge.mjs` `_handleServerMessage` (the `serverContent` branch), lines 1202–1216:

```js
const inputTranscription = serverContent?.inputTranscription || serverContent?.input_transcription;
if (inputTranscription?.text) {
  const txt = inputTranscription.text;
  // FORWARD ALL non-empty chunks. Gemini streams the USER's words in tiny pieces...
  if (txt && /[a-z0-9]/i.test(txt)) { this.onInputTranscript?.(txt); }
}
```

A prior fix deliberately removed a per-chunk `length >= 8` filter here (comment at 1205–1212) because it was dropping the small building-block deltas so the user's line never accumulated. Correct call — but it pushed **all** endpointing responsibility downstream to leo.mjs, which then endpoints wrongly.

**Accumulation + flush (the aggregation layer that already exists).** `leo.mjs`:
- `bridge.onInputTranscript` (3681–3703): appends each delta to the single buffer `bridge._inputTranscript`, resets a debounce timer, and flushes early on the punctuation/length triggers quoted in the TL;DR.
- `flushInputTranscript` (3443–3679): trims the buffer, drops <3-char noise (3451–3455), records the human voice event (3458–3464), runs self-echo suppression (3467–3501), narration-command intercept (3509–3514), AI/fleet-speech suppression (3525–3547), then posts `**${who} [Voice]:** ${spokenText}` to the resolved transcript channel (3549–3552) and arms the response watchdog (3554–3566).

**Why one utterance fragments.** Two compounding causes:
1. The early-flush triggers (3690–3692) fire on natural sub-phrase boundaries (7 words, a trailing `?`, a comma+capital), so a single continuous utterance is emitted as 2–4 turns.
2. The buffer is **global, not per-speaker** (`bridge._inputTranscript` is one string). If two people speak close together, or Gemini attributes partials to whoever `bridge._currentSpeaker` currently is, fragments can also cross-contaminate speakers.

### A.2 Proposed change and where it lands

Introduce a proper **endpointing/aggregation layer on the input side only**, replacing the eager flush with silence-based finalization:

1. **Silence-timeout endpointing.** Replace the "flush on 7 words / 48 chars / punctuation" logic in `bridge.onInputTranscript` (leo.mjs 3690–3696) with a rolling **inactivity timer** as the *primary* endpoint: each incoming delta resets a timer of `LEO_UTTERANCE_SILENCE_MS` (propose default 700–900ms, env-tunable). The utterance is only flushed when no new STT delta has arrived for that window. Keep a **max-utterance safety cap** (e.g. flush anyway at ~12s or ~400 chars) so a monologue still lands. Sentence-ending punctuation should *lengthen confidence* but no longer force an immediate flush on its own.
2. **Per-speaker buffering.** Change `bridge._inputTranscript` (single string) into a per-speaker map keyed by `bridge._currentSpeakerId` (the id already resolved at flush time, leo.mjs 3523). Each speaker gets its own buffer + its own silence timer, so overlapping speakers don't merge and a speaker change forces a flush of the previous speaker's buffer. This is the natural home for the "buffer partial STT fragments per speaker" requirement.
3. **Finalization signal, if available.** If Gemini Live emits a segment-final marker on `inputTranscription` (needs a one-session log capture to confirm — see Open Questions), prefer that as the endpoint and use the silence timer only as a fallback. Do this in the bridge (around 1203–1216) by passing an `isFinal` flag through `onInputTranscript(txt, isFinal)`; leo.mjs flushes immediately on `isFinal`, else runs the silence timer.

**Landing zones:** `leo.mjs` 3681–3703 (rewrite the flush-decision block), plus a small buffer-structure change touching every `bridge._inputTranscript` reference (2360, 3444–3447, 3682–3685, 5408, 5446). Optional bridge signature tweak at `gemini-live-bridge.mjs` 1202–1216.

### A.3 Risk + voice-out isolation

**Risk: LOW–MEDIUM.** This is purely input-text timing; it never touches `onAudioChunk`, the frame loop, `onTurnComplete` (1250–1273), or `onInterrupted` (1276–1283). The one real risk is **added latency**: a 700–900ms silence window delays when Leo "sees" the finished utterance, which could make him feel slower to respond. Mitigations: (a) make the window env-tunable and iterate with the owner; (b) keep the existing response watchdog (3554–3566) as the safety net but re-point it so it counts from utterance *finalization*, not from each fragment (today it can be re-armed by every fragment, which is part of the erratic timing). Do **not** shorten the window so far that it re-introduces fragmentation — this is an eyeball-iterate knob. Keep everything behind `LEO_UTTERANCE_SILENCE_MS` / a `LEO_UTTERANCE_ENDPOINTING=1` flag so it is instantly revertible.

---

## Workstream B — Stability / calibration

### B.1 What the real code does today

Four nested recovery layers exist (bottom-up):

1. **Launcher chain, dependency-gated at the top.** `Start-KAI.ps1` (stages 122–212) gates each stage on the previous being *ready*: Ollama → KAI engine `kai.exe --oracle` (waits up to `EngineTimeoutSec`≈300s for port :3334, `Start-KAI.ps1` 155–164, hard-throws if it never answers) → `kai_supervisor.py` (RAM watchdog, only after engine confirmed) → overnight pipeline → `run-oracle-discord.ps1 -NoStartKai`. `run-oracle-discord.ps1` (19–44, 302–342, 598) sets the sleep-list and calls `run-ecosystem.ps1`, which just runs `node ecosystem-manager.mjs`.
2. **In-tree supervisor: `ecosystem-manager.mjs`.** Owns fleet startup order but by **timer-stagger only, not readiness gating**: Dashboard, then Oracle (523–524), KAI after 2s (526–530), then Analyst/Researcher/Kai-Coder at +1200ms stagger (536–547), then Leo/Gemini/Claudey/X/Groq ("Leo first = voice anchor", 537, 548–561). `scriptForProcess()` maps names to scripts (106–113): Oracle→`oracle-gateway.mjs`, Leo→`bots/leo.mjs`, X/Claudey/Groq→`native-bot.mjs`, KAI→`bots/kai.mjs`, Dashboard→`command-center-server.mjs`. Respawn on `child.on('close')` (468–506): 5s re-spawn, with a **wedge guard** (`recordExitAndCheckWedged`, 80–86) that suspends auto-respawn after 3 exits in 60s. IPC restart paths (`RESTART_BOT`, `SLEEP/WAKE`, `PHOENIX_PROTOCOL`, `RESTART_ALL`) at 361–462; a file-queue fallback (`state/restart_requests.json`, polled every 5s, 581–598) fed by `process-supervisor.mjs queueRestartToFile()`.
3. **Per-process guards.** Leo's `unhandledRejection` / `uncaughtException` are **log-and-stay-alive** (leo.mjs 6639–6654) with a transient-net whitelist (6636–6637). Manager has the same (17–37). `ipc.mjs` handles `EADDRINUSE` by probing the port holder's `/health` and exiting 0 on self-duplicate/collision (151–200).
4. **Outermost watchdog: `phoenix-watchdog.ps1`** (Task Scheduler, every 5 min). 3-pulse check — engine :3334, manager PID + heartbeat <45s, dashboard :3001 (38–65). Crash-loop aware: ≥15 "exited with code" lines ⇒ restart just the engine (throttled once/8min, 105–129); manager-down ⇒ storm-guarded full relaunch honoring `authorized_stop.json` (131–232).

**Gemini Live reconnect** (`gemini-live-bridge.mjs` `ws.on('close')`, 926–1155): config remembered at connect (724–729, `_userClosed`). Cascade: model-rejected ⇒ rotate `LIVE_MODEL_FALLBACKS` (949–968); billing/rate-limit ⇒ per-model cooldown + key rotation through `_keyList`, then a single retry after `KAI_LIVE_BILLING_COOLDOWN_MS`≈15min (969–1073); **connectivity-aware** ⇒ if offline go dormant and rebuild via `connectivity.whenOnline()`, if online linear backoff `1000ms×attempt` capped at 3 tries then a 30s recovery poll (1074–1114). **Context preserved** via session-resumption handles (873–878, 1157–1163) and GoAway clean-reconnect (1165–1190). `connectivity.mjs` uses raw TCP probes (not DNS) with dormant-not-die doctrine.

**Other-API fallback.** Leo's neural path is dual-provider (leo.mjs 6463–6497): Groq → Ollama fallback (and vice-versa). Voice degrade: the bridge fires `onVoiceDegraded` when all Live keys are depleted (1047–1051) **but no bot registers that callback** — so on full depletion Leo goes silent for the cooldown instead of dropping to edge-TTS.

**Observers (diagnose, don't restart):** `heartbeat-monitor.mjs` isolates a dead bot for 5 min (`suppressBot`, 112–123); `resource-saver.mjs` is a CPU/RAM/GPU governor that throttles loops but never restarts.

### B.2 Gaps (mapped, from the real code)

- **No true dependency gating inside the manager** — Leo/socials spawn on fixed `setTimeout` offsets (539–561) with no check that Oracle/KAI actually finished booting. A slow Oracle boot lets dependents start against a not-ready parent.
- **Orphaned voice-degrade signal** — `onVoiceDegraded` emitted (1048) but unconsumed; Leo has no TTS-failover wiring.
- **Bridge reconnect ceiling** — 3 quick tries + one 30s poll while online (1093, 1108); if that fails and the socket never re-closes, voice stays down while the *process* is still "healthy," so nothing upstream restarts it.
- **Wedge guard can strand a bot** — 3 exits/60s ⇒ suspended with no auto-retry (496–501); a transient triple-crash needs human/Oracle action.
- **`shared/*.mjs` patches never auto-restart** (`process-supervisor.mjs` 58–59) — the bulk of the code base.
- **Engine is a hard SPOF** — its death crash-loops every bot on :3334; recovery only via phoenix-watchdog's ≥15-exit branch or `kai_supervisor.py`, up to ~5 min latency.

### B.3 Proposed change and where it lands

Do **not** rip out the layered design — it is mature. Close the specific gaps:

1. **Explicit dependency/readiness gating in `ecosystem-manager.mjs`** (539–561). Before spawning each dependent tier, poll the parent's `/health` (Oracle, KAI, Dashboard already expose health per B.1) with a bounded wait + timeout, replacing blind `setTimeout` stagger with "start when parent is ready, else after cap." Encode an explicit dependency table (`Leo depends on {Oracle, KAI, Dashboard}`) so the order is data, not timing luck.
2. **Wire the orphaned voice-degrade callback.** Register `bridge.onVoiceDegraded` in leo.mjs to flip Leo to the existing edge-TTS path (he already has `en-US`/`en-GB` edge voices) for the cooldown window, then auto-restore when Live keys recover. This is a *voice-out-adjacent* change → treat as MEDIUM risk (see B.4); gate behind a flag and iterate.
3. **Escalation after reconnect exhaustion.** When the bridge's online reconnect (1092–1114) exhausts its 3 tries + 30s poll, emit a distinct "voice-stuck" signal the supervisor/heartbeat can act on (restart just Leo), so a wedged socket isn't invisible to the health layer.
4. **Wedge-guard auto-retry with backoff.** Add a single delayed auto-clear attempt (exponential backoff) before requiring human action (496–501), so transient port contention self-heals.
5. **Codify startup/dependency + failure/recovery as a documented contract** (a short `FLEET-SUPERVISION.md`) so "what depends on what / what recovers each" is explicit, matching the owner's ask.

### B.4 Risk + voice-out isolation

**Risk: MEDIUM.** Items 1, 3, 4, 5 are supervisor/launcher-side and cannot affect audio playout. Item 2 (voice-degrade → TTS failover) is the only one that touches Leo's speaking path — it must reuse the *existing* edge-TTS engine (already proven) and must NOT alter `onAudioChunk` or the Live frame loop; it only chooses TTS when Live is fully unavailable. Flag-gate it (`LEO_TTS_FAILOVER=1`) and verify with the owner that switching mid-session doesn't stutter. Everything else is safe to ship independently.

---

## Workstream C — Transcript registration & verification

### C.1 What the real code does today

**Registration is a hardcoded 1:1 map, not the SQLite DB.** Three redundant copies of "which channel is whose":
- `channel-rules.mjs` 29 `USER_TRANSCRIPT_MAP` `{userId → transcriptChannelId}`; 37 `TRANSCRIPT_USER_INFO` reverse map `{channelId → {userId,name,role,slotIdx}}`.
- `identities.mjs` 6 `HUMAN_REGISTRY` carries `transcriptChannelId` + `voiceSlot` per human.
- `voice-manager.mjs` 7 `FIXED_ASSIGNMENTS` `{userId → slotIndex}`.
- Runtime mirrors in leo.mjs: `userTranscriptChannels` (Map userId→channelId, 1017), `userToSlot`/`slotToUser` (1014, 1040–1043).

**k (Ryan) and Taz are hardcoded:** `USER_TRANSCRIPT_MAP["1111106883135217665"] = "1500527640107417783"` (Ryan, Slot 1, channel-rules.mjs 30) and `["1286110163505385523"] = "1500529928184008885"` (Taz, Slot 2, 31), mirrored in identities.mjs and voice-manager.mjs. `getTranscriptChannel(userId)` (voice-manager.mjs 55) is a pure `USER_TRANSCRIPT_MAP[userId] || null` lookup.

**The SQLite store (`transcripts.db`, `transcript-memory.mjs`) is episodic memory, not the registration map.** `transcript_fts` (36) stores `speaker, user_id, content, context, channel_id, timestamp`; recall is keyed by `channel_id`/`user_id`. There is no `slots`/`registration` table.

**Slots: fixed cap of 6.** `channel-rules.mjs` 16 `LEO_VOICE_SLOTS` is a 6-element channel-ID array (Slot 1 Ryan, 2 Taz, 3–4 open/guest, 5–6 public). Dynamic assignment only fills idx 4–5: `assignSlot` loops `for (let i=4; i<6; i++)` and returns `-1` (Full) if both taken (voice-manager.mjs 76–87). On `-1`, leo.mjs drops the join: `No slots available for ${userId}. Ignoring.` (2100–2107).

**Speaker → channel resolution:** `resolveLiveTranscriptChannel` (leo.mjs 3433–3441): runtime `userTranscriptChannels.get(sid)` → `getTranscriptChannel(sid)` → fallback `CHANNEL_IDS.SUNDAY` (public).

**The "application sequence / gatekeeping" is essentially soft, not a hard gate.** What exists is `triggerVoiceLockOnboarding(user, profileName)` (leo.mjs 4598–4622), called from the voice-join handler (2213–2222) for any non-bot whose name isn't in `biometrics.profiles`. It posts a `[SECURITY ALERT]` to `CHANNEL_IDS.UNREGISTERED_SLOT` and DMs an **optional** "voice lock signature" enrollment script ("You can still use the system without this"). The real verification tech (`identity-vault.mjs` Azure `IdentityVault`) has a **stub `verifyIdentity`** (empty body). So enrollment is offered but nothing blocks speech; access to a transcript channel is just a Discord permission overwrite granted automatically (`updatePermissions` 104, `bootstrapPermissions` 148). No queue/waitlist/apply-to-speak exists (`applyToSpeak`/`waitlist` grep empty).

### C.2 Proposed change and where it lands

The owner wants: register everyone to a transcript channel **by user ID**; verify who's registered to whom and when; only use an open transcript that belongs to that registered ID; **retire** the soft "application/voice-lock sequence" as the default; and fall back to a sequence **only when all slots are full**.

1. **Make registration data, not code.** Add a persistent registration record keyed by user ID (extend the existing `data/voice_slots.json` used by `getSlotAssignments`/`assignSlot`, or a small `registrations` table in `transcripts.db`) storing `{userId, name, transcriptChannelId, slotIdx, registeredAt, registeredBy}`. Keep the hardcoded `USER_TRANSCRIPT_MAP` for k/Taz as seed data, but resolve dynamic users from the persistent store so "who is registered to whom and when" is queryable. `registeredAt` satisfies the "and when" requirement.
2. **Registration-verified resolution.** Tighten `resolveLiveTranscriptChannel` (leo.mjs 3433–3441): only return a channel if the resolved channel's registered owner ID matches the current speaker ID; never silently fall through to `SUNDAY` for a registered user (that fallback is a likely source of cross-user leakage). Add an explicit `verifyRegistration(userId, channelId)` helper.
3. **New default flow ("just converse → name → register").** When an unregistered person speaks and a slot is free: Leo converses normally, asks their name, calls a new `registerUser(userId, name)` that claims a free slot (reusing `assignSlot`, voice-manager.mjs 63–87) and writes the persistent record, then uses it exactly like k/Taz. Retire `triggerVoiceLockOnboarding` as the gate — keep the voice-signature enrollment as a *later, optional* security add-on, not an entry step.
4. **Slots-full fallback = the retained sequence.** Keep the current "application sequence" (the voice-lock / unregistered-slot path, plus the `assignSlot === -1` branch at leo.mjs 2100–2107) **alive but reachable only when `assignSlot` returns -1** (all dynamic slots taken). That is the owner's explicit "unless all transcript slots are full, fall back to the sequence" condition. Everywhere else, skip it.

### C.3 Risk + voice-out isolation

**Risk: MEDIUM.** All changes are on the transcript-routing / identity side, not audio. The genuine hazard is **mis-routing**: a bug in registration-verified resolution could send one person's words to another's channel (the exact corruption the current code warns about at leo.mjs 3519–3523). Mitigate by keeping the change additive and logged, verifying against real speakers with the owner, and never removing the `SUNDAY` fallback for *unregistered* speakers (only for registered ones where a wrong-owner match should hard-stop rather than mis-route). No audio-path code is touched.

---

## Workstream D — New-joiner onboarding + passive profile

### D.1 What the real code does today

- **There is NO server-join handler anywhere.** Grep for `guildMemberAdd`/`GuildMembers` across `bots/*.mjs`, root `*.mjs`, and `shared/*.mjs` returned **zero matches** (verified directly). The Discord clients don't register the `GuildMembers` intent.
- **Oracle's live entry is `oracle-gateway.mjs`** (ecosystem-manager.mjs 107), and it has **no `voiceStateUpdate`, no `member.send`, no join handler** (verified directly). Oracle only DMs the **owner** (`client.users.fetch(OWNER_ID)` → `owner.send`) for EOD reports (oracle-gateway.mjs 692/777/784). Oracle's message entry is `client.on('messageCreate')` (891) which resolves identity (`resolveIdentityFromMemory`, 897) and fire-and-forgets each message to `/api/digest-message` (901–922).
- **The DM primitive is proven, but in `index.mjs` (a separate voice client), not Oracle.** `index.mjs` voice-join branch (657–697) DMs first-timers: `member.send("**Oracle:** Welcome to the Roundtable. I've assigned you to Private Transcript #…")` (682) and the capacity-full variant (667). So `member.send(...)` works; it just isn't wired into Oracle or to server-join.
- **The "chat they can't reply in" is behavior, not a string.** No user-facing text says this (grep empty). It exists only as code comments (leo.mjs 1577, 1773; index.mjs 827) describing the fleet's listen-but-don't-respond behavior in social/work channels. A welcome/notice line would need to be authored.
- **Passive profile capture already exists — but on Leo/native-bot, not Oracle.** `profile-memory.mjs recordProfile()` (44) calls `captureEntityFacts()` (56) as a side-effect of recording, and posts a `[PROFILE]` line to the SENSITIVE channel. It is invoked for every non-bot human message in leo.mjs (1961–1962) and imported by native-bot.mjs (8). The extractor `captureEntityFacts` (transcript-memory.mjs 597–619) is conservative first-person regex (`/\bi\s+(?:love|adore)\s+(.+)/i` → `like`, etc., one fact/message, no LLM). Facts land in the `entity_facts` table; `getEntityProfile()` (627) returns a compact summary for prompt injection. A second store, `user-warehouse.mjs` (`user_facts`/`user_fingerprints`, keyed by user ID, `setFact` at 105), is defined but has **no live caller in the message hot path**.
- **Oracle writes no profiles** (grep for `recordProfile|setFact|captureEntityFacts` in oracle-gateway.mjs empty).

### D.2 Proposed change and where it lands

1. **Add a server-join handler.** Register a new `client.on('guildMemberAdd', …)` in Oracle (`oracle-gateway.mjs`) and add the `GuildMembers` intent to Oracle's client. On join: (a) post a **friendly welcome** to `CHANNEL_IDS.WORK` (used at oracle-gateway.mjs 936/1105/1237) — warm, not "you can't reply here"; and (b) `member.send(...)` a **concise DM tour** (reuse the proven `member.send` pattern from index.mjs 682) explaining what the server is and what to expect — short, friendly, and **without** saying "I'm building your profile."
2. **Author the copy, keep it short.** The welcome + DM tour text needs writing (there's no existing string to reuse). Keep the DM to a few sentences; the owner explicitly does not want a wall of text or a dead-end tone.
3. **Passive profile on Oracle's message path.** Import `recordProfile` and call it right after the digest in `oracle-gateway.mjs messageCreate` (after 922, before command routing at 929+), mirroring leo.mjs 1961 — it already has `message.author.id`, resolved `from`, `message.content`, `message.channelId` in scope. This gives Oracle the same passive, non-announced fact capture Leo has, satisfying "passively register useful details through normal conversation." No new store needed; `entity_facts` + `captureEntityFacts` already do this conservatively.
4. **Optionally enrich the extractor** later (reaction-triggered facts for E), but v1 should just reuse the existing conservative regex path to stay safe.

### D.3 Risk + voice-out isolation

**Risk: LOW.** Entirely text/DM/Discord-event side; no audio path involved. Risks are product-shaped, not stability-shaped: (a) DM spam / Discord rate limits if welcome fires repeatedly — guard with a "already welcomed" check against the registration store from C; (b) privacy tone — the DM must not read as surveillance, per the owner. `member.send` failures are already `.catch()`-safe in the existing pattern. Adding the `GuildMembers` intent is a config change on Oracle's client only.

---

## Workstream E — Vitals / emotion-aware context (theory-of-mind)

### E.1 What the real code does today

- **Vitals are computed in the Rust engine and read back via the metrics store.** `drive-system.mjs readEngineState()` (158–168) pulls `valence, chi, phi_g, rho` (aggregated over a window from `source='rust-engine'`). They're assembled and exposed by `command-center-server.mjs buildVitals()` (3089), served at **`:3001` GET `/api/vitals`** (3758–3766). The engine is at `:3334` (`/api/session`, `/api/status`); the KAI bot's in-process drives at `:3401` GET `/drives` (kai.mjs 643).
- **Drive/valence mapping.** `drive-system.mjs applyEngineSignals()` (173–218) maps engine vitals → six drives (satisfaction ← valence+coherence, prediction_error ← chi, curiosity ← 1−phi_g/5, etc.). Public API: `getDrives()` (493) raw 0–1 drives; `getDriveDirective()` (436–476) turns levels into prose directives (`[DRIVE: PAIN]…`, `[DRIVE: SOCIAL] Extended silence. Engage genuinely…`, else `[DRIVE: BALANCED]`); `getPredictionConfidenceDirective()` (520–530) hedges from prediction accuracy. A separate `autonomic-state.mjs getDrives()` (31) computes anxiety/curiosity/urgency from CPU/RAM/latency with its own `directive`. **No `NeuralOscillator` class exists in the JS tree** — "valence" originates in Rust.
- **Does any speaking bot read its own state at composition time?** Only partially, and **not the richest signal for Leo.** Leo's prompt builder `buildLeoSystemPrompt` (leo.mjs 4407; native-bot.mjs 2176) injects **only** `getPredictionConfidenceDirective()` (leo.mjs 4494, native-bot.mjs 2215). The social/text path `start-bot.mjs` (1382–1456) injects `autonomic-state` anxiety + prediction directive. **`getDriveDirective()` (the mood directive) and raw engine vitals are computed but never injected into any spoken prompt** — they only feed `/drives` and the dashboard.
- **Reading the other party's reaction: none.** Grep for `sentiment`/`userMood`/`detectEmotion`/`theoryOfMind`/`readReaction` empty. `metacognition.mjs` (62–63, 308–321) models other **bots** (`[EMPATHY]`), not the human; `getSnapReaction` (native-bot.mjs 2970, leo.mjs 5519) is Leo's own filler ack, not a read of the user. `epistemic-vault.mjs emotionalWeight` is memory salience. `biographies.mjs` calls Leo "emotionally perceptive" as persona claim with no backing detection.

### E.2 Proposed change and where it lands

The owner wants the speaking AI to factor in **its own current vitals/emotional state** plus **a read of the other party's reaction**, lightweight and only on turns that warrant it.

1. **Inject own-state into the prompt (the easy, high-value half).** In `buildLeoSystemPrompt` (leo.mjs 4407, at the existing hook line 4494) and its native twin (native-bot.mjs 2215), add `getDriveDirective()` and a compact vitals line (from the already-exposed `/api/vitals` or a cached `readEngineState()`), alongside the prediction directive that's already there. For Oracle, add the same at its prompt/status assembly (oracle-gateway.mjs 365–388). This immediately gives "factor in my own state before composing" using signals that already exist and are already rendered elsewhere — no new computation.
2. **Gate it so it's not on every message.** Only include the mood/vitals block when it "warrants it": e.g. when a drive directive is non-`BALANCED`, or on the first turn of a conversation, or every N turns, or when the vitals delta since last turn exceeds a threshold. This satisfies "lightweight, not on every message" and avoids prompt bloat.
3. **Reaction-read of the other party (the new half).** Add a lightweight `readReaction(userText, priorContext)` — start rule-based/cheap (punctuation, short affirmations vs. pushback, sentiment lexicon, response latency already available from the utterance-endpointing timer in A), optionally a tiny LLM classification reusing the `getSnapReaction` plumbing. Feed a one-line `[THEORY-OF-MIND: user seems …]` note into the same gated prompt block. Only run it on turns that warrant it (e.g. after Leo asked something, or when the user's utterance is emotionally loaded), not every message.
4. **Source of truth:** reuse `getDriveDirective`/`getDrives` (drive-system) and `/api/vitals` — do not invent a parallel emotion model.

### E.3 Risk + voice-out isolation

**Risk: LOW–MEDIUM.** This is prompt-context assembly, not audio — it does not touch `onAudioChunk` or the frame loop. Risks: (a) **prompt bloat / latency** in the Live session if the block is large or added every turn — hence the gating in item 2; (b) **behavioral drift** — feeding mood into Leo could make him moody/erratic in ways the owner dislikes (this is the "paced producer made it worse" family of risk, but on the text-prompt side). Mitigate: flag-gate (`LEO_SELF_STATE_IN_PROMPT=1`, `LEO_TOM=1`), keep the injected text terse and directive-style (matching the existing `[DRIVE: …]` format), and eyeball-iterate with the owner one signal at a time (own-state first, reaction-read second). Do not let vitals injection change turn timing or interrupt behavior.

---

## Suggested build order / phasing

**Phase 1 — A: utterance endpointing (highest leverage, most isolated).** Fix the fragmented-voice bug: silence-based endpointing + per-speaker buffering (leo.mjs 3681–3703), flag-gated, iterate the silence window with the owner. Ship alone so its effect on turn-taking is cleanly observable. Re-point the response watchdog to fire from finalization.

**Phase 2 — C: registration by ID.** Persist registrations, registration-verified `resolveLiveTranscriptChannel`, new "converse → name → register" default, retire the voice-lock sequence except on slots-full. Do this before D/E because both rely on clean per-user identity and "who is registered when."

**Phase 3 — B: supervision hardening.** Dependency/readiness gating in the manager, wire the orphaned `onVoiceDegraded` → TTS failover (the one voice-adjacent item — flag-gate + owner verify), reconnect-exhaustion escalation, wedge auto-retry, and the `FLEET-SUPERVISION.md` contract. Independent of A/C; slotted third because it's the largest surface and partly overlaps the fragile path.

**Phase 4 — D: onboarding + passive profile on Oracle.** Add the `guildMemberAdd` handler + `GuildMembers` intent, welcome-to-WORK + DM tour (author copy), and `recordProfile` on Oracle's messageCreate. Depends on C for the "already registered/welcomed" check.

**Phase 5 — E: vitals/theory-of-mind in prompts.** Inject own-state (drive directive + vitals) into the prompt builders, gated; then add the reaction-read. Last because it's an enhancement layered on a calibrated, stable base, and it's the highest behavioral-drift risk.

Each phase is independently shippable and flag-gated; none requires the next to function.

---

## Open questions — need an owner decision

1. **A — silence window value.** What `LEO_UTTERANCE_SILENCE_MS` feels right (700? 900? 1200?)? This is an eyeball-iterate knob; needs a live fly/talk session. Also: is a max-utterance cap of ~12s acceptable, or do you want longer monologues to land as one turn?
2. **A — does Gemini Live emit a segment-final flag?** I could not confirm from the code whether `inputTranscription` carries an `isFinal`/segment marker. One instrumented voice session (log the raw `inputTranscription` payloads) would tell us whether we can endpoint on a real signal instead of only silence. Worth doing before finalizing A.
3. **C — where should registrations persist?** Extend `data/voice_slots.json`, or add a `registrations` table to `transcripts.db`? And do you want registration to require your approval, or fully self-serve (Leo asks name → registers) for anyone in a free slot?
4. **C — keep 6 slots?** The cap of 6 (2 fixed + 2 guest + 2 public) is hardcoded in `LEO_VOICE_SLOTS`. Is 6 still right, or should the dynamic range grow before "slots full → sequence" triggers?
5. **B — voice-degrade → TTS failover.** Confirm you want Leo to auto-switch to edge-TTS when all Gemini Live keys deplete (vs. staying silent). This touches the speaking path, so it needs your sign-off and a live test.
6. **D — welcome + DM tour copy.** The exact wording needs your voice. Also: should the public welcome fire in `WORK`, or a dedicated welcome channel? And should the passive-profile capture on Oracle post `[PROFILE]` lines to the same SENSITIVE channel Leo uses, or somewhere else?
7. **D — GuildMembers intent.** Enabling `guildMemberAdd` requires the privileged `GuildMembers` gateway intent on Oracle's bot (and the toggle in the Discord Developer Portal). Confirm that's acceptable.
8. **E — how often should own-state / theory-of-mind fire?** "Turns that warrant it" needs a concrete rule: non-BALANCED drive only? Every N turns? After Leo asks a question? And how expressive should mood be in his replies — subtle, or clearly felt?
9. **E — reaction-read fidelity.** Start rule-based/cheap, or spend a small LLM call per warranted turn? The latter is richer but adds latency/cost to the Live loop.

---

*End of plan. No code was modified. Implementing any phase requires the usual surgical, flag-gated edits + a Codex version bump at that time.*