KAI Language "Best Setup" — Layered Morphological Encoder + Grammar Feature Lattice

_Design + proven prototype (see KAI-morph-probe.py). Goal: Kai KNOWS word structure &

grammar, LEARNS smaller units from observation, and the lattice carries what the

modules need. Saved 2026-09-21._

The five layers (bottom = robust/dumb, top = smart/grammatical)

• L0 Character n-grams (2,3,4) — universal fallback. Encodes ANY string incl.

irregulars (buzz, smog, fridge), typos, names, any language. Always on.

• L1 Learned sub-word vocabulary — morphemes (roots/prefixes/suffixes) INDUCED

from his own corpus by frequency (unsupervised morphology; BPE/Morfessor-style).

Grows as he observes more text. PROVEN: found able/ed/ible/ing/ion/ly/tion +

root "struct" with no hardcoded list.

• L2 Allomorph + rule normalization — a taught table + the 70 grammar rules map

variants to ONE unit (ible->able), tag part-of-speech, mark affix type. PROVEN:

able vs ible 0.500 -> 0.800; preferable vs impossible 0.166 -> 0.308.

• L3 Grammatical features on each lattice cell — every cell stores its

decomposition (stem + affixes), POS, and features (number / tense / person /

valency). This is the "built-in needs for the modules."

• L4 Modules consume L3 — order_by_corpus / word_calculus read POS+features; a

new agreement+argument-structure check gates assembly; grounded-generation composes

under those constraints and the corpus-fit gate.

What each of the owner's 70 rules maps to

• Morphology (prefix/suffix/root/inflection/allomorph) -> L1 + L2.
• Word classes / POS, agreement/concord, argument structure/valency -> L3 + the L4 check.
• Syntax / CFG / production / precedence -> the L4 assembler (upgrade of the carry math).
• Regex / n-grams / tokenization -> the INPUT (parse) side, sharpening L0/L1 segmentation.
• Phonology, medical, Semitic-specific -> not used at the current text stage.

Migration reality (honest, non-negotiable)

1. Changing the encoder invalidates every existing cell vector. The whole lattice

(~92k cells) must be RE-ENCODED once so old + new share one space. Batch job,

done offline, verified before swap.

2. The morpheme vocabulary is LEARNED once from his corpora ([internal module],

[internal module] exist) and thereafter updated as he observes.

3. All additive + flag-guarded + staged. Prototype first (done, in Python), then

Rust port of the encoder, then the re-encode, then the L3 cell fields, then the

L4 checks. Each step measured before the next.

Proven so far (KAI-morph-probe.py, open test)

able=ible unified (0.80), suffix sharing recovered, roots found, irregulars safe,

control (cat/dog) stays 0. This validates L0-L2. L3/L4 are next.