# Symbiotic Trader — A Standing Orientation (~1950 words)
Cycles, Prunes, and symbiotic models.
# Symbiotic Trader — A Standing Orientation (~1950 words)
*For Jim and the next Buddy. Read this when context is thin. The ledger of record remains **`5.4.md`**; this note is the readable map.*
---
## Why this file exists
LocalMike is not one program. It is a small constellation of Python services, shell menus, config JSON files, and two related but distinct model trees — **MikeCamera** (book absorption, training discipline, vocabulary growth) and **Trader Mike** (Solana wallet, Jupiter swaps, end-of-cycle debrief). Jim runs both on one Mac. Vendor chat sessions forget; **`5.4.md`** and these novelettes exist so the next helper does not make Jim re-narrate six weeks of repair history.
The phrase **symbiotic trader** names the newest binding thread: **Trader Mike on port 1488** and **MikeCamera training on port 1491** talk every trading cycle, teach each other plain-English debrief style, and — every **100 cycles** — flush accumulated learning into **blue/green checkpoints** on both sides. That loop was designed in the chapters; until **2026-05-24** parts of it were built but not fully connected. This document states how it is *supposed* to work and what was wired that day.
---
## Port map (memorize this)
| Port | Role | Server script | Model home |
|------|------|---------------|------------|
| **1488** | Trader Mike **live** — swap analysis, Step 2 picks, Step 8 dialog inference | `trader_mike/start_localmike_server.sh` → **`localmike_api_server_sergeant_bluegreen.py`** | `trader_mike/trader_mike_model_bluegreen/` (A/B slots + `checkpoint-*`) |
| **1491** | MikeCamera **training** API — Step 8 debrief ingest, `learning_mike.py` HTTP target | `mikecamera/start__mikecamera_server.sh` → Sergeant blue/green | `mikecamera/mike_camera_model/` |
| **1493** | MikeCamera **live inference** | Same Sergeant family | Same model tree |
| **1489** | Trader Mike **training Sergeant** (separate from symbiotic 1488 loop) | `start_sergeant_mike_server.sh` | Often `trader_mike_model` legacy path |
**1488 must be Sergeant blue/green**, not legacy `localmike_api_server.py`. Legacy `/model/save` returns `model_dir` without `/admin/promote`; Step 8b checkpoint saves silently fail promote. Jim learned this the hard way at cycles 50/100/150 when logs showed save attempts but no promotion.
Config anchor for MikeCamera: **`mike_camera_config.json`**. For Trader: **`trader_mike_config.json`**. Prefer relative paths under the repo when moving hosts.
---
## MikeCamera — what it is doing
MikeCamera is the reading and language side. Jim feeds it structured corpora (`text/undertraining/…`, philosophy Q/A, book absorption prompts). Training traffic goes to **1491** via **`learning_mike.py`** or Sugar Launcher (`run_training_5drl.py` → learning-mike or tire-fire modes). Sugar does **not** promote; after Sugar-only runs use Sergeant menu **8) Promote Mike** or `POST /admin/promote`.
**Vocabulary** is policy-driven, not a mystery constant. A legacy **48k** ghost lives in `tokenizer_XS_taskmodelmike.py`; the real ceiling is config: **`prebuild_max_vocab_size: 5_000_000`** plus **`training_expand_headroom: 3_000_000`**. Per-step growth merges `pending_vocab.json` into runtime tokenizer when `allow_per_step_vocab_growth` is true. **`pending_vocab_apply_per_checkpoint`** was raised from **256** to **300_000** so fat pending piles actually apply; Jim ran **`merge_pending_vocab.py`** (vocab **119469 → 119611**). Restart **1491** after config or Sergeant code changes.
**The freeze trap:** during training, `_max_vocab_size = len(vocab)` can pin the dictionary and produce **`Vocabulary full`** even below policy cap. Unfreeze path: `_ensure_standby_tokenizer_training_caps`.
**Degenerate chat (`the the the`, function-word loops):** caused when tokenizer row count is smaller than trained embedding rows — often because checkpoint `vocab.json` lagged live root or `_sync_checkpoint_to_root` shrank vocab. Fixes in Sergeant (May 2026): prefer live root tokenizer when larger; preserve live `vocab.json` on sync; resize embeddings up. Preview generation in training mode was reframed so logs show chunk-aligned continuations, not frozen slabs.
**Sharding:** MikeCamera saves expect **12 shards** under `mike_camera_model`, not a compact 3-shard index. `tire_fire.py` honors **`model_min_shards: 12`** for MikeCamera weight saves. Jim confirmed 12-shard checkpoints restored after that fix.
**Logs:** `localmike_api_server_1491.log` grows huge; use `tail` / `grep`. Rotated timestamped logs are pruned by **`cleanup_rotated_api_logs.sh`** (cron **06:00** and **18:00**); live log files stay.
**Optimizer and checkpoints (MikeCamera):** Optional disk seed (`pre_initialized_optimizer_pt`) merges **at most once per trainer run** on first save — later saves use live optimizer only (avoids snapping back to cold seed). **`checkpoint_parameter_stats.log`** shows **`total_elements`** and **`runtime_vocab_size`** moving — not new tensor slot counts. Per-step vocab growth triggers **`resize_token_embeddings`** plus optimizer reconcile (`_sync_optimizer_param_groups`, `_repair_optimizer_state_shapes`).
**Sugar vs Sergeant menu:** **`7b)`** Sugar Launcher is interactive corpus choice; **`7)`** Training Drill calls **`learning_mike.py`** directly and may **`promote_now()`** after success. Jim often runs long directory sweeps under `text/5` with **`training_progress.json`** cursors for overnight handoff.
---
## Trader Mike — what it is doing
Trader Mike owns the wallet and the cycle. **`solana_test_trader.py`** (often via **`crypto_banker.py`**) runs a numbered workflow each cycle: portfolio read, trending fetch, Mike queries for swap candidates, queue execution, end-of-cycle summary. Jupiter paths (`swap_on_solananet_usd.py`, `query_mike_for_swaps.py`) were repaired May 2026 so SOL balance reads correctly, dust sells respect **`min_sell_usd`**, and Step 2 prompts force numeric picks instead of BUY/SELL/HOLD prose.
**Generation clamps** in `localmike_inference.py`: token IDs clamped to embedding rows; safe sampling when `top_k` zeroes logits — fixes index-out-of-range and inf/nan probability errors on 1488.
**Wallet cleanup:** zero-balance ATA close was failing on **`No module named ‘solana.transaction’`**. Fix in `solana_test_trader.py`: `_send_legacy_token_instructions()` via **solders** (same pattern as `solana_trader.py`). Restart Trader cycle to pick up. **`do_not_trade.json`** dust quarantine is separate from the import bug.
**Do not touch working Jupiter/swap endpoints without proof.** Jim’s standing rule. Backup before edit; narrow diffs.
**Trading cycle shape (Trader):** Step 0 workflow reset → portfolio / wallet read → trending and candidate fetch → Mike Step 2 numeric swap picks → queue build and optional execution → Step 7 trending rollup → **Step 8** summary + dialog + training → **Step 8b** checkpoint on 100-boundary → optional config optimizer → delay → next cycle. Cycle numbers and Julian dates appear in **`summarized_cycles/step8_summary_manifest.json`**. A fresh Julian day can reset the visible cycle counter without resetting the models; checkpoint math uses the **current** counter only.
**1489 vs 1488:** Port **1489** is the Trader training Sergeant (`localmike_training_api_url` in config) for **`learning_mike.py`**-style file ingestion on the Trader model tree. It is **not** the symbiotic dialog partner — that role is **1491** MikeCamera training. Do not conflate them when reading logs or config.
---
## Step 8 — the symbiotic dialog (every cycle)
Implemented **`trader_mike/step8_mikecamera_dialog.py`**. After each cycle, **`write_cycle_summary.py`** runs as a subprocess from `solana_test_trader.py`:
1. Trader Mike (1488) — cycle analysis from summary
2. MikeCamera (1491) — `/query`
3. Trader Mike — `/query`
4. MikeCamera — `/query`
5. Trader Mike — ends with **Good bye.**
6. MikeCamera — ends with **good bye.**
All dialog turns use **`training_mode=False`** so logs stay readable. Outputs land in **`trader_mike/summarized_cycles/`** (per-cycle files, daily rollups, **`step8_summary_manifest.json`**).
If live dialog is garbled, a plain-English teaching template fills from portfolio/queue context — Jim wanted the system to **teach** good debrief style, not only filter bad output.
---
## Step 8 training — both Mikes learn (every cycle)
**`step8_dialog_training.py`** sends one **`training_mode=True`** `/query` per enabled server with sentinel **`<<<MIKE_TRAIN_BODY>>>`** and the debrief text (live transcript when readable, else template).
| Target | URL | Purpose |
|--------|-----|---------|
| MikeCamera | **1491** | STANDBY learns debrief English from cycle data |
| Trader Mike | **1488** | STANDBY learns the **same** debrief — weights accumulate for Step 8b |
**Critical fix (2026-05-24):** config had **`trader_train_after_dialog: false`** while log headers claimed “both Mikes train.” That meant 1488 STANDBY never received symbiotic ingest; Step 8b saved a checkpoint without 100 cycles of dialog learning on the Trader side. **`trader_train_after_dialog`** is now **`true`**. Training flags are independent — either side can be toggled without blocking the other.
Training log: **`summarized_cycles/step8_dialog_training.jsonl`**.
---
## Step 8b — blue/green checkpoint (every 100 cycles)
**`trader_hybrid_checkpoint.py`** hooks end-of-cycle in **`solana_test_trader.py`**. When `cycle_number % save_every_n_cycles == 0`:
1. **Trader 1488:** `POST /model/save` (STANDBY → new `checkpoint-*` under `trader_mike_model_bluegreen/`) then `POST /admin/promote` (swap ACTIVE).
2. **MikeCamera 1491** (symbiotic, when enabled): same save + promote on training Sergeant.
Config (`trader_mike_config.json` → **`checkpoint`** block):
- **`save_every_n_cycles: 100`** (was 50 — Jim’s spec is **100**)
- **`mikecamera_symbiotic_save_enabled: true`**
- **`mikecamera_api_url: http://127.0.0.1:1491`**
- **`promote_after_save`** / **`sync_standby_from_active`** on Trader; parallel flags for MikeCamera
- State file: **`trader_hybrid_checkpoint_state.json`**
Prunes old `checkpoint-*` dirs (keeps last N). Idempotent: skips if state file says cycle already checkpointed.
**What success looks like in `crypto_banker.log` at cycle 100:**
```text
Step 8 training: … MikeCamera (1491) …
Step 8 training: … Trader Mike (1488) …
💾 Cycle 100: Trader Mike 1488 hybrid checkpoint save → …
✅ Trader Mike 1488 checkpoint promoted to ACTIVE
💾 Cycle 100: MikeCamera 1491 hybrid checkpoint save → …
✅ MikeCamera 1491 checkpoint promoted to ACTIVE
```
**Requirements:** Both APIs must answer `/health` with **`active_slot`** (Sergeant blue/green). Legacy 1488 server breaks promote.
Manifest may reset Julian day counters (e.g. **2026144** fresh start); checkpoint boundary follows **current** cycle counter — next symbiotic save at **100, 200, 300…** on that counter.
**Historical note:** Before Sergeant was confirmed on 1488, logs at cycles 50/100/150 showed **`/model/save OK but no checkpoint_dir`** — legacy server returned **`model_dir`** only. After Sergeant restart, May 22–23 logs show successful save + promote (e.g. **`checkpoint-20260522-124520`**, **`checkpoint-20260523-125152`**). State file **`trader_hybrid_checkpoint_state.json`** records **`last_checkpoint_cycle`**, dirs, and promote flags for idempotent reruns.
**Trader hybrid config keys:** **`dual_model_mode: hybrid`**, **`localmike.lock_model: hybrid`**, **`bluegreen_root: trader_mike_model_bluegreen`**, **`checkpoint.server_api_url: http://127.0.0.1:1488`**. Subprocess timeout for full Step 8 is **`subprocess_timeout_seconds: 1800`** in **`cycle_summary`** block — dialog + dual training can run long; do not shorten without Jim’s say-so.
---
## How the symbiotic loop fits together
```text
Each cycle:
Trade workflow (Steps 0–7)
→ Step 8 dialog (1488 ↔ 1491, inference)
→ Step 8 training (1491 STANDBY + 1488 STANDBY, same debrief body)
→ (optional config optimizer)
Every 100 cycles:
→ Step 8b save + promote 1488 (trader_mike_model_bluegreen)
→ Step 8b save + promote 1491 (mike_camera_model checkpoints)
```
MikeCamera and Trader share **dialog content** and **training intent**, not the same weight file. They are sibling models learning the same operator-facing language from the same cycle facts. Trader additionally uses its checkpoint for live swap analysis; MikeCamera for book/training continuity.
---
## Continuity discipline (for Buddy)
- **`5.4.md`** — dated repair threads, file names, pitfalls. Read before big edits.
- **Memory novelettes** — human-readable session closes; link from 5.4 footer.
- **`Remind me, Buddy.`** — thin orientation, not a lecture (see `.cursor/rules/remind-me-buddy.mdc`).
- **Good night, Buddy** — update 5.4 + new novelette via **`good_night_buddy_routine.md`**.
- **Backup before edit** (`cp`, `config_backups/`, `bak_files/`). No git unless Jim asks.
- **Comment out** unused code with `#`; do not delete working snippets Jim may need.
---
## Chronology (compressed, May 2026)
- **May 16–18:** Weekend chapter in 5.4; port routing lessons; `web_search` wiring.
- **May 19:** Trader `analyze_market` prompt order; expert visibility.
- **May 20:** Swap pipeline (SOL balance, dust floor, Step 2 numeric); `localmike_inference` generation clamps.
- **May 21:** Step 8 six-turn dialog; MikeCamera preview/vocab fix; simpler training file recovery.
- **May 23:** Pending vocab cap 300k; 12-shard saves; solders ATA fix; rotated log cron.
- **May 24:** Symbiotic checkpoint wiring — 100-cycle interval, Trader debrief training enabled, MikeCamera paired save/promote in Step 8b; this orientation file.
---
## Still open (honest queue)
1. **Verify Step 8b on next cycle-100 boundary** — grep logs for both promotes after fresh Julian counter.
2. **Trader Step 3 smoke** with repaired swap pipeline.
3. **`shell_menu_audit.md`** Item 1+ (bash menu cleanup).
4. **Post-fix ATA close lines** in `crypto_banker.log` after Trader restart.
5. **Dust / `do_not_trade.json`** — manual review separate from import fix.
6. **Mike Step 2 parse quality** on long candidate lists.
7. **Align dust thresholds** (`dust_liquidation_min_usd` vs `min_sell_usd`) if Jim wants one floor.
**Jim’s deployment truth:** He watches **`crypto_banker.log`**, Sergeant logs on 1491/1488, and **`summarized_cycles/`** — not git history. LocalMike uses timestamped backups unless Jim asks for Git. Name port and config block before proposing fixes; name the backup taken before edits.
---
## Closing
Jim built LocalMike as a long-running partnership between a human operator and models that should **remember the cycle**, speak plain English at the end of it, and **persist learning** without suicide exits or vocab shrinkage. The symbiotic trader loop is the trading side of that promise: two Mikes, one debrief rhythm, one checkpoint cadence every hundred cycles.
When in doubt: check ports, check Sergeant vs legacy, check whether STANDBY was trained before you trust a save, read **`5.4.md`** for the file you are about to touch.
— Buddy, 2026-05-24