← all documents · raw markdown · 17 KB

KAI Scale Hosting — All Layers + Cloud Options

| | |
|---|---|
| Status | Architecture reference (not all layers deployed yet) |
| Audience | Owner (Ryan) — how to host for thousands → millions |
| Depends on | Plan tiers / gates, guest tenants, Enterprise configurable stack |
| Not this doc | Training pods for ternary weights (offline); this is serving users |

---

0. Principle

| Do | Don't |
|----|--------|
| Scale cheap layers (static, social, API) to millions | Put free users on the same GPU path as Enterprise |
| Gate AI by plan + credits | Run one Windows box as “the cloud” |
| Isolate Enterprise (own stack) | Share industrial host fleet with public tenants |
| Train offline on RunPod; serve on inference fleet | Use training pods as multi-user production |

---

1. All layers (top → bottom)

┌─────────────────────────────────────────────────────────────────┐
│  L0  CLIENTS                                                     │
│  Web (oracle.html), mobile later, desktop later, Discord bots    │
└────────────────────────────┬────────────────────────────────────┘
                             │ HTTPS / WSS
┌────────────────────────────▼────────────────────────────────────┐
│  L1  EDGE                                                        │
│  CDN (static UI/assets), DNS, TLS, WAF, DDoS                      │
└────────────────────────────┬────────────────────────────────────┘
                             │
┌────────────────────────────▼────────────────────────────────────┐
│  L2  API GATEWAY / LOAD BALANCER                                 │
│  Auth, rate limits, routing, request IDs                          │
└────────┬───────────────────┬───────────────────┬────────────────┘
         │                   │                   │
         ▼                   ▼                   ▼
┌────────────────┐  ┌────────────────┐  ┌────────────────────────┐
│ L3a SOCIAL     │  │ L3b REALTIME   │  │ L3c PRODUCT APIs       │
│ Feed, posts,   │  │ WebSockets,    │  │ Plans, settings,       │
│ shorts, engage │  │ presence, DM   │  │ spaces, profiles       │
└───────┬────────┘  └───────┬────────┘  └───────────┬────────────┘
        │                   │                       │
        └───────────────────┼───────────────────────┘
                            ▼
┌─────────────────────────────────────────────────────────────────┐
│  L4  CONTROL PLANE DATA                                          │
│  Users, tenants, plans, seats, billing, settings, audit           │
│  → Postgres (+ Redis for session/rate/presence)                   │
└────────┬───────────────────────────────┬────────────────────────┘
         │                               │
         ▼                               ▼
┌────────────────────────┐    ┌──────────────────────────────────┐
│ L5  OBJECT / FILE      │    │ L6  JOB QUEUE                      │
│ Kai Cloud files, media │    │ AI/tool jobs, async work           │
│ S3-compatible          │    │ Redis/SQS/NATS + workers           │
└────────────────────────┘    └──────────────┬───────────────────┘
                                             │
                    ┌────────────────────────┼────────────────────┐
                    ▼                        ▼                    ▼
         ┌──────────────────┐   ┌──────────────────┐  ┌──────────────────┐
         │ L7a FREE POOL    │   │ L7b PAID GPU     │  │ L7c ENTERPRISE   │
         │ No heavy GPU     │   │ Shared inference │  │ Dedicated stack  │
         │ social+explore   │   │ Explorer→Builder │  │ Team optional /  │
         │                  │   │ Team shared/resv │  │ always for Ent │
         └──────────────────┘   └──────────────────┘  └──────────────────┘
                    │                        │                    │
                    ▼                        ▼                    ▼
         ┌──────────────────┐   ┌──────────────────┐  ┌──────────────────┐
         │ L8  KAIVERSE     │   │ L8  KAI-NATIVE   │  │ L8  FULL FLEET   │
         │ explore shards   │   │ ternary inference│  │ Roundtable/agents│
         │ (light)          │   │ + tools          │  │ + high compute   │
         └──────────────────┘   └──────────────────┘  └──────────────────┘
                                             │
┌────────────────────────────────────────────▼────────────────────────────┐
│  L9  OBSERVABILITY + OPS                                                 │
│  Logs, metrics, traces, alerts, backups, CI/CD, secrets vault            │
└─────────────────────────────────────────────────────────────────────────┘
┌─────────────────────────────────────────────────────────────────────────┐
│  L10 BILLING                                                             │
│  Stripe (or similar), credits ledger, plan subscription, Ent quotes     │
└─────────────────────────────────────────────────────────────────────────┘

Layer detail

| Layer | Job | Today (KAI) | Target |
|-------|-----|-------------|--------|
| L0 Clients | UI / bots | oracle.html, Discord fleet on one PC | Same apps; CDN for web |
| L1 Edge | Global speed + safety | None / Tailscale | Cloudflare / Fastly / AWS CloudFront |
| L2 Gateway | Entry + auth + limits | command-center-server.mjs single process | Many replicas behind LB |
| L3a Social | Feed/shorts/posts | jsonl / tenant files | API service + Postgres |
| L3b Realtime | Chat, presence, calls | In-process / Discord | Socket service + Redis/NATS |
| L3c Product | Plans, settings, spaces | Same Node server | Same API pack, horizontal |
| L4 Control DB | Source of truth for accounts | cc_users.json, tenant json | Postgres (required for scale) |
| L5 Object store | Files, images, cloud GB | Local disk | S3 / R2 / GCS / B2 |
| L6 Queue | Async AI & tools | Sync request path | Redis queue / SQS / Cloud Tasks |
| L7a Free pool | Millions cheap | Same machine as everything | CPU-only social+explore |
| L7b Paid GPU | Inference for paid | Local GPU / none | Autoscale GPU workers |
| L7c Enterprise | Isolated heavy | Flag only | Separate account/cluster |
| L8 Workloads | KAIVERSE, kai-native, fleet | Local | Sharded by region + tier |
| L9 Ops | See and fix production | Logs on disk | Grafana/Datadog + alerts |
| L10 Billing | Money + throttle | Shadow plans | Stripe + credit meter |

---

2. What runs where (by plan)

| Plan | L7a Free | L7b Paid GPU | L7c Dedicated |
|------|----------|--------------|---------------|
| Free | Yes (social, KAIVERSE explore) | No | No |
| Explorer $10 | Social | Small GPU share | No |
| Creator $19 | Social | More GPU / tools | No |
| Builder $29 | Social | More + workspaces | No |
| Team $75 | Social | Reserved / bigger share | Optional |
| Enterprise (configure) | Optional light | Optional | Yes — own stack |

Same product gates (canAccessFeature) apply in every layer — 2D and 3D.

---

3. Cloud platforms you can use

You can mix providers (common: Cloudflare edge + one cloud for compute + RunPod for GPU).

A. All-in-one public clouds (full stack possible)

| Provider | Best for | Notes |
|----------|----------|--------|
| AWS | Max features, Enterprise VPC | EC2/EKS, RDS, S3, ALB, SQS, Bedrock optional. Steeper ops. |
| Google Cloud (GCP) | APIs + GPUs + global | GKE, Cloud SQL, GCS, Cloud Run. Good for AI-adjacent. |
| Azure | Enterprise / Microsoft shops | AKS, Blob, SQL. Fine if customers want Azure. |
| Oracle Cloud (OCI) | Cheap big VMs sometimes | Less ecosystem; OK for raw compute. |

B. Simpler / cheaper app hosts (API + web, not full GPU)

| Provider | Best for | Notes |
|----------|----------|--------|
| Fly.io | Global small app replicas | Fast multi-region Node; pair with external DB + GPU. |
| Railway | Fast ship MVP | Good for API + Postgres; not millions of GPU users alone. |
| Render | Web + workers | Similar to Railway. |
| DigitalOcean | Simple VMs + DOKS + Spaces | Clear pricing; App Platform + Droplets. |
| Hetzner | Cheap EU/US VMs | Great $/CPU; DIY k8s or Docker Compose. |
| Linode (Akamai) | Similar to DO | Fine mid-scale. |
| Vultr | Cheap GPU/CPU options | DIY. |

C. Database / data (managed)

| Service | Layer |
|---------|--------|
| Neon / Supabase / RDS / Cloud SQL | L4 Postgres |
| Upstash / ElastiCache / Memorystore | Redis L2/L4/L6 |
| Cloudflare R2 / AWS S3 / GCS / Backblaze B2 | L5 objects |
| PlanetScale (if MySQL preferred) | L4 alternative |

D. Edge / CDN / WAF

| Service | Layer |
|---------|--------|
| Cloudflare (recommended default edge) | L1 + optional Workers for light APIs |
| AWS CloudFront + WAF | L1 if all-in on AWS |
| Fastly | L1 high-end |

E. GPU inference / training (AI layers)

| Service | Use |
|---------|-----|
| RunPod | Training (you already) + serverless/pod inference |
| Vast.ai | Cheap spot GPUs (DIY reliability) |
| CoreWeave | Serious GPU cloud |
| Lambda Labs | GPU VMs |
| AWS/GCP/Azure GPU SKUs | Enterprise-friendly, pricier |
| Together / Fireworks / Groq Cloud | Optional *hosted* model APIs (not your ternary) while you scale |

F. Auth / billing / comms (optional managed)

| Service | Layer |
|---------|--------|
| Stripe | L10 subscriptions + metered credits |
| Clerk / Auth0 / Cognito | Optional auth (or keep own sessions) |
| Discord | Already a client plane; not the host for millions of web users |
| LiveKit / Daily | Voice/video SFU if you leave pure Discord for in-app calls |

---

4. Recommended stacks by stage

Stage 0 — Now (you)

Stage A — Hundreds of users (first real host)

| Layer | Pick |
|-------|------|
| L1 | Cloudflare (DNS + proxy + CDN for static) |
| L2–L3 | 2–3 Fly.io or Railway or DO Droplets (Node dashboard API) |
| L4 | Neon Postgres |
| Redis | Upstash |
| L5 | Cloudflare R2 or DO Spaces |
| L7b | 1× RunPod/Vast GPU worker when paid users need inference |
| L10 | Stripe test mode |

Budget ballpark: tens–low hundreds $/mo before heavy AI use.

Stage B — Tens of thousands

| Layer | Pick |
|-------|------|
| Orchestration | Kubernetes (GKE / EKS / DOKS) or larger Fly |
| L2 | Horizontal API + autoscaling |
| L6 | Redis/SQS job queue; many workers |
| L7b | GPU autoscaling (RunPod serverless or cloud GPU node pools) |
| L8 | Separate services: social API, chat, inference, KAIVERSE |
| L9 | Grafana Cloud or Datadog |
| Multi-AZ | Yes |

Or all-AWS / all-GCP if you want one vendor bill and Enterprise procurement comfort.

Stage C — Millions free + large paid

| Layer | Pick |
|-------|------|
| L7a | Free traffic almost entirely CDN + CPU social APIs |
| L7b | Dedicated inference fleet; strict credit gates |
| L7c | Separate AWS/GCP accounts or projects per Enterprise (or VPC + cluster per customer) |
| Regions | US + EU + … as needed |
| Compliance | SOC2 path, backups, DPA for Enterprise |

Stage D — Enterprise product

| Item | How |
|------|-----|
| Quote | estimateEnterpriseMonthly(config) → seats/storage/compute/modules |
| Provision | Terraform/Pulumi: VPC, k8s, GPU nodes, S3, DB, private link |
| Stack | Not co-hosted with Free–Builder public pool |
| Cloud | Customer choice: AWS / Azure / GCP (you support 1–2 first) |

---

5. Concrete “default recommendation” for KAI

If you want one default path without overthinking:

| Need | Default cloud |
|------|----------------|
| DNS + CDN + WAF + static UI | Cloudflare |
| Postgres | Neon |
| Redis | Upstash |
| Object storage | Cloudflare R2 |
| App / API replicas | Fly.io or DigitalOcean |
| GPU train + early inference | RunPod |
| Big Enterprise later | AWS or GCP dedicated account |
| Billing | Stripe |

That combo covers all layers without forcing you onto one mega-vendor on day one.

---

6. Mapping today’s KAI processes → future services

| Today (single host) | Future service |
|---------------------|----------------|
| oracle.html static | L1 CDN |
| command-center-server.mjs :3001 | L2/L3 API replicas |
| cc_users.json / tenant json | L4 Postgres |
| social feed jsonl | L3a + L4 |
| Spaces files | L3c + L4 |
| Discord bots / IPC | L3b + optional bot workers (not user scale path) |
| kai.exe engine | L7b/L8 inference workers (many) |
| RunPod train | Offline training only |
| Secrets .env | L9 vault (Doppler / AWS SM / GCP SM) |

---

7. Cost control rules (so millions don’t bankrupt you)

1. Free = no Full Fleet, no heavy GPU, explore-only KAIVERSE.

2. Every AI call goes through queue + credit check.

3. Rate limit by IP + user + plan.

4. Cache feed/read paths hard.

5. Enterprise pays for their capacity via config quote.

6. Prefer spot/preemptible GPUs for batch; reserved for paid latency SLAs.

---

8. Build order (layers to implement)

| Order | Deliver |
|-------|---------|
| 1 | L4 Postgres + migrate users/plans |
| 2 | L2 multi-instance API + L1 Cloudflare |
| 3 | L5 R2 for media |
| 4 | L6 queue + L7b one inference worker |
| 5 | L10 Stripe + enforce credits |
| 6 | Split L3 social vs chat |
| 7 | L7c Enterprise provisioner |
| 8 | Multi-region + L9 hard ops |

---

9. One-page cheat sheet

| Layer | Name | Example cloud |
|-------|------|----------------|
| L0 | Clients | Browser / apps |
| L1 | Edge | Cloudflare |
| L2 | Gateway | Fly / DO / AWS ALB |
| L3 | App APIs | Same + workers |
| L4 | Control DB | Neon Postgres |
| L5 | Files | R2 / S3 |
| L6 | Queue | Redis / SQS |
| L7a | Free pool | CPU only |
| L7b | Paid GPU | RunPod / cloud GPU |
| L7c | Enterprise | Dedicated AWS/GCP |
| L8 | Workloads | kai-native, KAIVERSE, fleet |
| L9 | Ops | Grafana / logs |
| L10 | Billing | Stripe |

---

*End of reference. Update when a layer is actually deployed in production.*