# KAI Scale Hosting — All Layers + Cloud Options | | | |---|---| | **Status** | Architecture reference (not all layers deployed yet) | | **Audience** | Owner (Ryan) — how to host for thousands → millions | | **Depends on** | Plan tiers / gates, guest tenants, Enterprise configurable stack | | **Not this doc** | Training pods for ternary weights (offline); this is **serving users** | --- ## 0. Principle | Do | Don't | |----|--------| | Scale **cheap layers** (static, social, API) to millions | Put free users on the same GPU path as Enterprise | | **Gate** AI by plan + credits | Run one Windows box as “the cloud” | | **Isolate** Enterprise (own stack) | Share industrial host fleet with public tenants | | Train offline on RunPod; **serve** on inference fleet | Use training pods as multi-user production | --- ## 1. All layers (top → bottom) ``` ┌─────────────────────────────────────────────────────────────────┐ │ L0 CLIENTS │ │ Web (oracle.html), mobile later, desktop later, Discord bots │ └────────────────────────────┬────────────────────────────────────┘ │ HTTPS / WSS ┌────────────────────────────▼────────────────────────────────────┐ │ L1 EDGE │ │ CDN (static UI/assets), DNS, TLS, WAF, DDoS │ └────────────────────────────┬────────────────────────────────────┘ │ ┌────────────────────────────▼────────────────────────────────────┐ │ L2 API GATEWAY / LOAD BALANCER │ │ Auth, rate limits, routing, request IDs │ └────────┬───────────────────┬───────────────────┬────────────────┘ │ │ │ ▼ ▼ ▼ ┌────────────────┐ ┌────────────────┐ ┌────────────────────────┐ │ L3a SOCIAL │ │ L3b REALTIME │ │ L3c PRODUCT APIs │ │ Feed, posts, │ │ WebSockets, │ │ Plans, settings, │ │ shorts, engage │ │ presence, DM │ │ spaces, profiles │ └───────┬────────┘ └───────┬────────┘ └───────────┬────────────┘ │ │ │ └───────────────────┼───────────────────────┘ ▼ ┌─────────────────────────────────────────────────────────────────┐ │ L4 CONTROL PLANE DATA │ │ Users, tenants, plans, seats, billing, settings, audit │ │ → Postgres (+ Redis for session/rate/presence) │ └────────┬───────────────────────────────┬────────────────────────┘ │ │ ▼ ▼ ┌────────────────────────┐ ┌──────────────────────────────────┐ │ L5 OBJECT / FILE │ │ L6 JOB QUEUE │ │ Kai Cloud files, media │ │ AI/tool jobs, async work │ │ S3-compatible │ │ Redis/SQS/NATS + workers │ └────────────────────────┘ └──────────────┬───────────────────┘ │ ┌────────────────────────┼────────────────────┐ ▼ ▼ ▼ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │ L7a FREE POOL │ │ L7b PAID GPU │ │ L7c ENTERPRISE │ │ No heavy GPU │ │ Shared inference │ │ Dedicated stack │ │ social+explore │ │ Explorer→Builder │ │ Team optional / │ │ │ │ Team shared/resv │ │ always for Ent │ └──────────────────┘ └──────────────────┘ └──────────────────┘ │ │ │ ▼ ▼ ▼ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │ L8 KAIVERSE │ │ L8 KAI-NATIVE │ │ L8 FULL FLEET │ │ explore shards │ │ ternary inference│ │ Roundtable/agents│ │ (light) │ │ + tools │ │ + high compute │ └──────────────────┘ └──────────────────┘ └──────────────────┘ │ ┌────────────────────────────────────────────▼────────────────────────────┐ │ L9 OBSERVABILITY + OPS │ │ Logs, metrics, traces, alerts, backups, CI/CD, secrets vault │ └─────────────────────────────────────────────────────────────────────────┘ ┌─────────────────────────────────────────────────────────────────────────┐ │ L10 BILLING │ │ Stripe (or similar), credits ledger, plan subscription, Ent quotes │ └─────────────────────────────────────────────────────────────────────────┘ ``` ### Layer detail | Layer | Job | Today (KAI) | Target | |-------|-----|-------------|--------| | **L0 Clients** | UI / bots | `oracle.html`, Discord fleet on one PC | Same apps; CDN for web | | **L1 Edge** | Global speed + safety | None / Tailscale | Cloudflare / Fastly / AWS CloudFront | | **L2 Gateway** | Entry + auth + limits | `command-center-server.mjs` single process | Many replicas behind LB | | **L3a Social** | Feed/shorts/posts | jsonl / tenant files | API service + Postgres | | **L3b Realtime** | Chat, presence, calls | In-process / Discord | Socket service + Redis/NATS | | **L3c Product** | Plans, settings, spaces | Same Node server | Same API pack, horizontal | | **L4 Control DB** | Source of truth for accounts | `cc_users.json`, tenant json | **Postgres** (required for scale) | | **L5 Object store** | Files, images, cloud GB | Local disk | S3 / R2 / GCS / B2 | | **L6 Queue** | Async AI & tools | Sync request path | Redis queue / SQS / Cloud Tasks | | **L7a Free pool** | Millions cheap | Same machine as everything | CPU-only social+explore | | **L7b Paid GPU** | Inference for paid | Local GPU / none | Autoscale GPU workers | | **L7c Enterprise** | Isolated heavy | Flag only | Separate account/cluster | | **L8 Workloads** | KAIVERSE, kai-native, fleet | Local | Sharded by region + tier | | **L9 Ops** | See and fix production | Logs on disk | Grafana/Datadog + alerts | | **L10 Billing** | Money + throttle | Shadow plans | Stripe + credit meter | --- ## 2. What runs where (by plan) | Plan | L7a Free | L7b Paid GPU | L7c Dedicated | |------|----------|--------------|---------------| | Free | Yes (social, KAIVERSE explore) | No | No | | Explorer $10 | Social | Small GPU share | No | | Creator $19 | Social | More GPU / tools | No | | Builder $29 | Social | More + workspaces | No | | Team $75 | Social | Reserved / bigger share | Optional | | Enterprise (configure) | Optional light | Optional | **Yes — own stack** | Same product gates (`canAccessFeature`) apply in **every** layer — 2D and 3D. --- ## 3. Cloud platforms you can use You can mix providers (common: **Cloudflare edge + one cloud for compute + RunPod for GPU**). ### A. All-in-one public clouds (full stack possible) | Provider | Best for | Notes | |----------|----------|--------| | **AWS** | Max features, Enterprise VPC | EC2/EKS, RDS, S3, ALB, SQS, Bedrock optional. Steeper ops. | | **Google Cloud (GCP)** | APIs + GPUs + global | GKE, Cloud SQL, GCS, Cloud Run. Good for AI-adjacent. | | **Azure** | Enterprise / Microsoft shops | AKS, Blob, SQL. Fine if customers want Azure. | | **Oracle Cloud (OCI)** | Cheap big VMs sometimes | Less ecosystem; OK for raw compute. | ### B. Simpler / cheaper app hosts (API + web, not full GPU) | Provider | Best for | Notes | |----------|----------|--------| | **Fly.io** | Global small app replicas | Fast multi-region Node; pair with external DB + GPU. | | **Railway** | Fast ship MVP | Good for API + Postgres; not millions of GPU users alone. | | **Render** | Web + workers | Similar to Railway. | | **DigitalOcean** | Simple VMs + DOKS + Spaces | Clear pricing; App Platform + Droplets. | | **Hetzner** | Cheap EU/US VMs | Great $/CPU; DIY k8s or Docker Compose. | | **Linode (Akamai)** | Similar to DO | Fine mid-scale. | | **Vultr** | Cheap GPU/CPU options | DIY. | ### C. Database / data (managed) | Service | Layer | |---------|--------| | **Neon** / **Supabase** / **RDS** / **Cloud SQL** | L4 Postgres | | **Upstash** / **ElastiCache** / **Memorystore** | Redis L2/L4/L6 | | **Cloudflare R2** / **AWS S3** / **GCS** / **Backblaze B2** | L5 objects | | **PlanetScale** (if MySQL preferred) | L4 alternative | ### D. Edge / CDN / WAF | Service | Layer | |---------|--------| | **Cloudflare** (recommended default edge) | L1 + optional Workers for light APIs | | **AWS CloudFront + WAF** | L1 if all-in on AWS | | **Fastly** | L1 high-end | ### E. GPU inference / training (AI layers) | Service | Use | |---------|-----| | **RunPod** | Training (you already) + serverless/pod **inference** | | **Vast.ai** | Cheap spot GPUs (DIY reliability) | | **CoreWeave** | Serious GPU cloud | | **Lambda Labs** | GPU VMs | | **AWS/GCP/Azure GPU SKUs** | Enterprise-friendly, pricier | | **Together / Fireworks / Groq Cloud** | Optional *hosted* model APIs (not your ternary) while you scale | ### F. Auth / billing / comms (optional managed) | Service | Layer | |---------|--------| | **Stripe** | L10 subscriptions + metered credits | | **Clerk / Auth0 / Cognito** | Optional auth (or keep own sessions) | | **Discord** | Already a client plane; not the host for millions of web users | | **LiveKit / Daily** | Voice/video SFU if you leave pure Discord for in-app calls | --- ## 4. Recommended stacks by stage ### Stage 0 — Now (you) - **Where:** Windows host + Tailscale - **Layers live:** L0 + single process L2–L3 + local files - **GPU train:** RunPod only ### Stage A — Hundreds of users (first real host) | Layer | Pick | |-------|------| | L1 | Cloudflare (DNS + proxy + CDN for static) | | L2–L3 | **2–3** Fly.io or Railway or DO Droplets (Node dashboard API) | | L4 | **Neon** Postgres | | Redis | **Upstash** | | L5 | **Cloudflare R2** or DO Spaces | | L7b | 1× RunPod/Vast GPU worker when paid users need inference | | L10 | Stripe test mode | **Budget ballpark:** tens–low hundreds $/mo before heavy AI use. ### Stage B — Tens of thousands | Layer | Pick | |-------|------| | Orchestration | **Kubernetes** (GKE / EKS / DOKS) or larger Fly | | L2 | Horizontal API + autoscaling | | L6 | Redis/SQS job queue; many workers | | L7b | GPU autoscaling (RunPod serverless or cloud GPU node pools) | | L8 | Separate services: social API, chat, inference, KAIVERSE | | L9 | Grafana Cloud or Datadog | | Multi-AZ | Yes | **Or all-AWS / all-GCP** if you want one vendor bill and Enterprise procurement comfort. ### Stage C — Millions free + large paid | Layer | Pick | |-------|------| | L7a | Free traffic almost entirely CDN + CPU social APIs | | L7b | Dedicated inference fleet; strict credit gates | | L7c | **Separate AWS/GCP accounts or projects per Enterprise** (or VPC + cluster per customer) | | Regions | US + EU + … as needed | | Compliance | SOC2 path, backups, DPA for Enterprise | ### Stage D — Enterprise product | Item | How | |------|-----| | Quote | `estimateEnterpriseMonthly(config)` → seats/storage/compute/modules | | Provision | Terraform/Pulumi: VPC, k8s, GPU nodes, S3, DB, private link | | Stack | **Not** co-hosted with Free–Builder public pool | | Cloud | Customer choice: AWS / Azure / GCP (you support 1–2 first) | --- ## 5. Concrete “default recommendation” for KAI If you want one default path without overthinking: | Need | Default cloud | |------|----------------| | DNS + CDN + WAF + static UI | **Cloudflare** | | Postgres | **Neon** | | Redis | **Upstash** | | Object storage | **Cloudflare R2** | | App / API replicas | **Fly.io** or **DigitalOcean** | | GPU train + early inference | **RunPod** | | Big Enterprise later | **AWS** or **GCP** dedicated account | | Billing | **Stripe** | That combo covers **all layers** without forcing you onto one mega-vendor on day one. --- ## 6. Mapping today’s KAI processes → future services | Today (single host) | Future service | |---------------------|----------------| | `oracle.html` static | L1 CDN | | `command-center-server.mjs :3001` | L2/L3 API replicas | | `cc_users.json` / tenant json | L4 Postgres | | social feed jsonl | L3a + L4 | | Spaces files | L3c + L4 | | Discord bots / IPC | L3b + optional bot workers (not user scale path) | | `kai.exe` engine | L7b/L8 inference workers (many) | | RunPod train | Offline training only | | Secrets `.env` | L9 vault (Doppler / AWS SM / GCP SM) | --- ## 7. Cost control rules (so millions don’t bankrupt you) 1. **Free** = no Full Fleet, no heavy GPU, explore-only KAIVERSE. 2. Every AI call goes through **queue + credit check**. 3. **Rate limit** by IP + user + plan. 4. **Cache** feed/read paths hard. 5. Enterprise pays for **their** capacity via config quote. 6. Prefer **spot/preemptible** GPUs for batch; reserved for paid latency SLAs. --- ## 8. Build order (layers to implement) | Order | Deliver | |-------|---------| | 1 | L4 Postgres + migrate users/plans | | 2 | L2 multi-instance API + L1 Cloudflare | | 3 | L5 R2 for media | | 4 | L6 queue + L7b one inference worker | | 5 | L10 Stripe + enforce credits | | 6 | Split L3 social vs chat | | 7 | L7c Enterprise provisioner | | 8 | Multi-region + L9 hard ops | --- ## 9. One-page cheat sheet | Layer | Name | Example cloud | |-------|------|----------------| | L0 | Clients | Browser / apps | | L1 | Edge | Cloudflare | | L2 | Gateway | Fly / DO / AWS ALB | | L3 | App APIs | Same + workers | | L4 | Control DB | Neon Postgres | | L5 | Files | R2 / S3 | | L6 | Queue | Redis / SQS | | L7a | Free pool | CPU only | | L7b | Paid GPU | RunPod / cloud GPU | | L7c | Enterprise | Dedicated AWS/GCP | | L8 | Workloads | kai-native, KAIVERSE, fleet | | L9 | Ops | Grafana / logs | | L10 | Billing | Stripe | --- *End of reference. Update when a layer is actually deployed in production.*