# Learning Mike — school run notes (2026-06-02)
**Jim · Buddy handoff**
**Corpus:** `text/school/banks-richard_the-last-class.txt` (*The Last Class*, Richard Banks)
**Trainer:** `learning_mike.py` → **:1493** (MikeCamera book / Sergeant STANDBY)
**Prompt deck:** `prompts/training_prompts.txt` (absorption `Prompt:` lines — instruction only, not corpus)
---
## Epigraph (from the run)
> *”Nowadays we have a world of people, all the same, all dull, all safe and healthy and secure. Then it was a world of persons.”*
And from the training log, a fair question:
> **”Don’t anybody get along anymore?”**
> — Jim, watching CE loss that never hugs zero
---
## Short answer: no convergence to zero is not proof that learning is broken
**Cross-entropy loss on a ~201k-token vocabulary is not supposed to march toward 0** during a few epochs of book ingestion on a checkpoint that already carries dialog/math garbage priors.
| What people expect | What the system actually optimizes |
|--------------------|-------------------------------------|
| Loss → 0 = “perfect” | Mean CE per token has a **floor** near **ln(vocab) ≈ 12.2** for random guessing on 201k tokens |
| One school story fixes Mike’s speech | Slow nudge of weights on **STANDBY**; **ACTIVE** / Step 8 / Trader paths are separate |
| Epoch 2 should dive to single digits | Epoch 2 **rewinds the file**; loss restarts on the **same hard task**, not from scratch |
**Observed on this run (healthy):**
- `step_updated=True` on acknowledged chunks
- Epoch 1 on *Last Class*: roughly **44–52**, easing toward **~40**
- Epoch 2 from line 0: **~41 → high-39s / low-40s** (e.g. chunks 7–10 around **38.6–40.0**)
- That is **modest improvement**, not stagnation — the meter is just calibrated for a huge vocab and long chunks
**Do not stop a good run** only because the number is not near zero.
---
## Why `learning_mike.py` feels like it “never converges”
### 1. The reported number is not “per-token perplexity you can compare to textbooks”
Sergeant returns a **training step loss** for a **whole chunk** (~2048 characters of prose + masked guidance + one rotating `Prompt:` line). It is **not** the same as a clean average CE you’d plot for a small LM on MNIST-of-language.
### 2. Vocabulary size dominates the scale
With **~201,261** embedding rows, even a **well-behaved** mean CE often lives in a band that looks “high” compared to small-vocab runs (e.g. older logs near **22 → 12** on ~22k-scale setups). Comparing today’s **40** to April’s **12** is comparing different rulers.
### 3. Starting weights are not a blank slate
ACTIVE/STANDBY load from **merged symbiotic / dialog-era** checkpoints. The model already “knows” math-token and forum junk patterns. School prose fights that prior; loss stays elevated while weights move slowly.
### 4. Learning rate and one book are intentionally gentle
`training.learning_rate` = **5e-5**, **2 epochs** on one short story — by design **stability over dramatic loss collapse**. That is appropriate for production STANDBY on :1493; it is not a bug that loss does not plummet.
### 5. Gradient norms are loud; updates still land
:1493 logs show **very high gradient norm warnings** (vs `max_grad_norm_for_update=8`). Training continues; `max_loss_for_update=190000` is not blocking steps. Noisy gradients + small LR ⇒ **slow drift**, not fast convergence.
### 6. Fixes already in place (2026-06-02) — leave running code alone while a run is healthy
| Issue | Fix |
|-------|-----|
| `training_prompts.txt` trained as raw corpus | Prompt deck → **masked instruction**; only book body is trained |
| Server exit after `/model/save` | `learning_mike` **restarts** :1493 after save before next file/epoch |
| Wrong file order / resume confusion | `_partition_book_absorption_training_files()` |
---
## What “working” looks like for this mission
1. **Logs:** `Chunk N training_loss=… step_updated=True` — steady acks, no connection refused after save/restart.
2. **Trend:** Within an epoch, occasional **high-30s**; epoch 2 not worse than epoch 1 on average.
3. **After save + promote (later):** Short **:1493** `/query` in plain English — not erf/math soup (Step 8 may still need its own path).
4. **Not required for this run:** Loss near zero, or readable debrief on :1491 mid-epoch.
---
## When to actually intervene (not now)
| Signal | Action |
|--------|--------|
| `step_updated=False` repeatedly | Inspect STANDBY / optimizer / server 503 |
| Loss **>55** and rising every chunk for a full epoch | Consider lower `learning_rate` or shorter chunks |
| Chunks fail / :1493 down after save | Restart `start__mikecamera_server.sh start 1493`; confirm restart-after-save in `learning_mike.py` |
| Epoch 2 finishes flat at ~50 with no high-39s | Then we diagnose — not while high-39s are appearing |
**While acks continue and loss wanders high-30s / low-40s: do not touch working training.**
---
## Architecture reminder (ports)
| Port | Role |
|------|------|
| **1493** | Book / `learning_mike` STANDBY training (*Last Class* now) |
| **1491** | Step 8 dialog / symbiotic merged inference |
| **1488** | Trader live trading (meme keys, swaps — separate from school loss) |
Training the book on **1493** does not automatically fix **:1491** debrief garbage in the same afternoon. Same family of weights over time, different jobs.
---
## Command reference (this run)
```bash
cd /Users/sfccranberry/pythia/LocalMike/mikecamera/mikecamera
export LOCALMIKE_TRAINING_API_URL=http://127.0.0.1:1493
export LOCALMIKE_API_URL=http://127.0.0.1:1493
/Users/sfccranberry/pythia/LocalMike/local_mike/bin/python3 learning_mike.py \
--train-only \
--train-file prompts/training_prompts.txt \
--train-file text/school/banks-richard_the-last-class.txt \
--train-format raw \
--train-seconds 1800 \
--train-epochs 2 \
--no-background-learning
```
After a good epoch: `curl -X POST http://127.0.0.1:1493/model/save` — expect server exit; client restarts before continuing.
---
## Buddy’s verdict
**Superior work from Jim:** running the right corpus on the right port, catching the prompt-deck vs book distinction, and refusing to panic-stop a live run that is **acknowledging chunks and creeping down**.
**Zero is too far away** because it was never the destination. **Persons** — and models — learn in messy, incremental steps. Miss Hippiness would call that a cozy bit of restful safety between every bit of violence in the loss curve.
---
*Written 2026-06-02. Companion: `5.4.md`, `learning_mike.log`, `localmike_api_server_1493.g`.*