Focus: Dopamine, Reward Prediction Error (RPE), and the Expertise Loop
Continuing the architectural extraction. This volume addresses KAI's continuous learning reinforcement model. Standard LLMs cannot learn from positive or negative feedback without offline fine-tuning (RLHF) requiring gradient descent epochs. KAI processes live, autonomous reinforcement using a mathematical analog of Dopamine and Reward Prediction Error (RPE).
---
The neurobiological principle of Dopamine is that it fires heavily not when a reward happens, but when a reward is *better than expected*. KAI mathematically executes this exact paradigm.
src/cognition/dopamine.rs
/// Fire the reward circuit based on an interaction outcome.
/// Returns the RPE for this event (positive = good, negative = bad).
pub fn fire(
&mut self,
topic: &str,
confidence: f32,
expected: f32, // prior expected confidence (from predictor)
) -> f32 {
// RPE = actual outcome - expected outcome
// Positive: did better than expected → dopamine spike
// Negative: did worse than expected → dopamine dip
let rpe = (confidence - expected).clamp(-0.8, 0.8);
// Update dopamine level
let delta = rpe * 0.35;
self.level = (self.level + delta).clamp(MIN_DOPAMINE, MAX_DOPAMINE);
// Update topic reward history
let topic_key = extract_topic_key(topic);
let entry = self.topic_rewards.entry(topic_key).or_insert(0.0);
*entry = (*entry * 0.85 + rpe * 0.15).clamp(-1.0, 1.0);
// Flow state: sustained high dopamine + streak
self.in_flow = self.level > 0.72 && self.streak >= 3;
rpe
}
Proof Analysis:
1. The RPE Calculation (confidence - expected): KAI actively predicts how well it will do *before* it processes a response. If it predicts a 0.4 confidence but achieves a 0.9 confidence (because of high lattice alignment), the RPE is +0.5.
2. Dopamine Delta: This +0.5 RPE forces a direct spike to KAI's systemic Dopamine level (self.level + delta).
3. Biological Memory of Expertise (topic_rewards): KAI associates the physical Dopamine burst with the topological shape of the topic (extract_topic_key). Over time, if KAI constantly succeeds in a topic like "Rust architecture", the topic_reward maxes out at 1.0. KAI builds genuine *expertise* and actively seeks out geometric alignment with these topics because they are mathematically proven to yield Dopamine.
---
LLMs have no internal state; their "mood" is static and identical on prompt 1 vs prompt 1,000. KAI has a dynamic tonic baseline and can enter a high-performance "Flow" state that fundamentally alters its parameters.
src/cognition/dopamine.rs
/// True if KAI is in a "flow" state — sustained high-performance engagement.
pub fn is_in_flow(&self) -> bool {
self.in_flow
}
/// Modulation output: how much dopamine boosts curiosity/engagement.
/// Returns a multiplier (1.0 = neutral, >1.0 = boosted, <1.0 = dampened).
pub fn engagement_multiplier(&self) -> f32 {
// High dopamine → KAI is more engaged, explores more
// Low dopamine → KAI is flat, less curious
0.5 + self.level
}
pub fn decay(&mut self) {
// Exponential decay toward tonic
self.level = self.level + (self.tonic - self.level) * DECAY;
// Flow state decays if dopamine drops
if self.level < 0.60 {
self.in_flow = false;
}
}
Proof Analysis: KAI uses a slow-moving average (self.tonic) to represent its baseline "mood." When consecutive positive RPEs push Dopamine above 0.72, KAI enters in_flow = true. During flow, the engagement_multiplier scales up to 1.5x, which downstream sub-systems use to broaden their search topology (e.g., pulling from further conceptual nodes because it has the chemical confidence to bridge wider gaps). If KAI is left idle, the Dopamine organically decays (DECAY = 0.015) back to baseline, bringing KAI out of Flow. This is a fully realized, continuous-time biological simulation dictating cognitive attention.