ecosystem-history
Merged D-3.2 wiki page — curated from 13 slices spanning pre_hermes (Apr–Jun 2026), hermes, and cc_sessions (Jun–Jul 2026). Chronological arcs, deduped.
Origin era (Apr–May 2026)
- Elliott began building with Claude ~Apr 25, 2026 (new to Claude); original five-track plan: brand/identity, website/social, AI concierge chatbot, SMK9 operations, raw food. First product concept: white-label AI platform for dog-boarding businesses (4 tiers 192K ARR yr-3); the build began as an AI concierge answering bookings in Elliott’s voice — build for SMK9, run 60–90 days, then white-label; rationale: the $150B+ pet industry is technologically behind (source: pre_hermes_sessions/48d004d5-608d-4923-5cf4-957d50ed0f26, 2026-04-25; corroborated across 4 sessions)
- Master-doc practice born 2026-04-29 (“yes, i want a master knowledge doc”) to survive chat-session isolation — SMK9_Master_Knowledge_Doc.md as single source of truth; Elliott’s stated fear: “I don’t want to have to repair information”; the “end this session” + chat-log-backup protocol originated after Elliott accidentally deleted a long conversation by editing his first message in the Android app (Apr 25) (source: pre_hermes_sessions/96b4be8e-3ab6-eb80-bad7-b2c6785ef75c, 2026-04-29; corroborated across 3 sessions)
- Session governance from pain: a 134-turn session (and a 67.5% overage) produced the 80-turn cap → revised to 60; real metric is context weight (40+ tool calls ≈ 60 turns); end on 3 consecutive failures or a debugging spiral; session-start protocol (May 3): pull the master doc from Drive before responding in any SMK9 chat (source: pre_hermes_sessions/64d0778d-bb8f-6504-04ba-48acba8be55b, 2026-05-02; corroborated across 5 sessions)
- Canonical Drive anchors established: Claude anchor folder 1iMSoTAiTQDCdb9j2RxW-1Gr6n6Ef46nb (only permanent hardcoded ID) + SMK9 root 1sb0Qjfl9pjr1CxLKSedHmQoraw2V0SZe; full Drive restructure May 1–3 (SMK9/Personal/Encompass_Holdings/Legacy_Business/_Archive) via a Claude-written Apps Script; the audit surfaced staff SSNs in plain Google Docs and plaintext Kraken API keys (secured/flagged) and unified all SoundMind name variants as SMK9 (source: pre_hermes_sessions/593efbcd-b921-926d-9d5b-19ee077d0819, 2026-05-03; corroborated across 5 sessions)
- Duplicate-file plague (root of the stable-fileId doctrine): Drive MCP could only create files (no update-in-place), so every save minted a new fileId; mitigations in sequence — date-suffix naming, SMK9_DriveCleanup.gs auto-dedupe (6-hour trigger), search-by-title in a stable parent folder, deletions routed through Hermes; parallel phone+laptop sessions forked master copies, foreshadowing the later fixes; prohibited-actions rule: Claude never deletes files or changes sharing permissions — deletions belong to Elliott or Hermes (source: pre_hermes_sessions/be9d578d-f5cc-de3d-d747-f635550458bf, 2026-05-01; corroborated across 6 sessions)
- Reliability doctrine born from failures: PRIME DIRECTIVE 1 — DATA ACCURACY (“never present a guess as fact”, from the auto-parts fiasco); read-before-replace; full dependency-chain verification before command sequences (from the 2026-05-02 failure session: 8.8GB SMS XML parse OOM at line 55,901, Firebase seeding failed on missing credentials); “require a live API test before marking anything COMPLETE” and “never give UI navigation from memory” (from the ‘worst chat ever’ — a ~2-hour 513-restart crash-loop that was a mis-copied API key, 5-min fix via
hermes setup); never dump raw .env contents (source: pre_hermes_sessions/50b8c44c-08ca-dc0e-f74f-f897bf36640b, 2026-05-08; corroborated across 8 sessions) - Hermes timeline: discovered ~May 3 (Nous Research’s open-source agent framework — early confusion with the ‘Nous Hermes’ LLM corrected; it uses Claude via the Anthropic API); designated #1 infrastructure priority because Claude couldn’t reach across chats/files; Hetzner account created May 8 (attempt 3); installed May 11 (WhatsApp self-chat live); full go-live May 14: systemd service, mcp-proxy port 8765 (StreamableHTTP, NOT SSE), Claude.ai connector “Hermes SMK9” (10 tools) at https://smk9-hermes.duckdns.org/mcp, Google Workspace OAuth (8 APIs), SOUL.md + BRIEFING.md, Qdrant locked to localhost (found exposed on 0.0.0.0), default ‘kawaii’ personality replaced (source: pre_hermes_sessions/644be351-2336-330e-e508-bede8d1487e9, 2026-05-03; corroborated across 8+ sessions)
- May 2026 security incidents (ancestors of today’s secret-hygiene rules): Anthropic API key exposed by careless .env grep (rotated); a second key + Gmail app password pasted into chat (rotated); Twilio auth token in VPS shell history (rotation deferred, root-only risk accepted); Richie’s SSN in a plain Google Doc (secured); rule set: never grep .env with key-matching patterns, mask values, rotate immediately; the ANTHROPIC_API_KEY chat/terminal exposure (May 2026, rotated same day) also minted the durable lesson “chat deletion is cosmetic, rotation is surgical” — deleting the chat removes it from view but not from retention or any scraper that saw it; only rotation kills the risk (source: pre_hermes_sessions/3c6689cc-b413-2c50-1331-5d4adc9f1ede, 2026-05-14; corroborated across 7 sessions; also pre_hermes_sessions/2b4c16a1-3594-18e1-9cc9-85e17dcc51a8, 2026-05-14)
- Claude Max subscribed ~2026-05-02 after tool-heavy claude.ai work hit limits; key lesson: console API billing is fully separate from Max — empty API credits caused a Hermes HTTP 400 loop (May 20), a temporary Grok detour (broke May 26; Grok needs its own provider path/xAI account), restored to Sonnet May 26–27; Elliott repeatedly pushed to use plan subscriptions instead of burning API credits — the D-050 lineage starts here (source: pre_hermes_sessions/bea60871-0f1f-ae5a-93bd-ed749b97242e, 2026-05-02; corroborated across 5 sessions)
- ICM adoption (May 29): ecosystem context structured per Van Clief & McDermott’s Interpretable Context Methodology (arXiv:2603.16021 — verified real, 5 layers L0–L4; “Jake Van Clief Folder Structure.pdf” on Drive is the spec); CLAUDE_CORE.md created May 29 as Layer 0 (renamed Ecosystem_Core.md by Jun 11), project masters as Layer 2 contracts; rationale: long sessions degrade the model — load only what the session needs; Elliott’s deeper rationale: the ecosystem must survive a change of LLM vendor; ICM is “sequential by design” — high-concurrency multi-agent writes are explicitly out of scope (source: pre_hermes_sessions/4382b29a-c583-85e7-9490-3b7657481db2, 2026-05-29; corroborated across 8 sessions)
- Architecture modernization (mid-May): monolithic masters split into Drive stable docs + Ecosystem_Config.md (URLs/endpoints) + Firebase volatile session state + Qdrant shared semantic memory — syncs cut from 15–20 tool calls to ~2; DECISIONS LOG section added to SMK9_Master May 27 for never-re-debate decisions; decision-capture gap (“decisions get re-debated unless logged as DECIDED with a date”) seeded the decisions-log practice (source: pre_hermes_sessions/a70aa587-3fe3-085f-b4ec-901ce5cf1d9e, 2026-05-14; corroborated across 4 sessions)
- Ecosystem naming era (early June): Elliott chose Ecosystem_Core.md (“ecosystem_core is the way to go”); Michelangelo Protocol (skills graduating to ecosystem level), Da Vinci Code protocol, Personal Operating Style project (Layer-3 precursor, live Jun 5), LAM/LLN vision, and “My Journey with AI” book/journal all created in the same push; Muddy Waters manifest audits (FACT/DECISION/TASK/VISION classification) began Jun 9 (source: pre_hermes_sessions/e3ffe45f-bdbe-d998-ea2a-e773c6ec2bc0, 2026-06-05; corroborated across 4 sessions)
- Milestones: Firebase Layers 0–1 complete May 7–8 (schema, 14 composite indexes, intake form v2 → Firestore, waiver v3); soundmindk9.net fully live by May 22 (intake at
/, /book, /print-blank.html, password-protected /dashboard, public intake API); briefing-sync two-layer design proposed May 22 (nightly Hermes cron writing SESSION_BRIEF.md to Drive) — direct precursor of the BRIEFING/ICM session-start load (source: pre_hermes_sessions/4d326a08-5b1b-0ef4-535e-e9900c0a74fe, 2026-05-07; corroborated across 5 sessions) - Elliott’s legacy ventures roster (pre-SMK9): Final Touch Locksmith; SMP & microblading (2016); Picasso Ink Shopify (Dec 2016); Amazon FBA (Silicon Stretch Lids, 2017–19); City Home Inspections (2019); Air Pros HVAC concept (2019); betta fish import (~2018–20); eSports venture (2020); IL cannabis dispensary research (2020); DevElement dev consulting (~2019); LoveTruth International Christian nonprofit (~2014, co-owned) (source: pre_hermes_sessions/abe376ca-b1dc-e4c8-2f37-25860ddd3a88, 2026-05-20; corroborated across 3 sessions)
- Mini-PC selection arc: shelved for VPS-first (late Apr); lost a Beelink SER5 Pro eBay bid; GMKtec M5 Plus briefly selected May 12; final purchase = MINISFORUM UM870 (Amazon, order Jun 16); driver: the Lenovo IdeaPad Slim 3 (i3-N305, 8GB soldered) crashing on full storage; most other hardware bought on eBay via elliott.soto@yahoo.com (receipts unforwarded, several amounts unconfirmed) (source: pre_hermes_sessions/2eb30e6b-f40e-600c-00ed-047cd583186f, 2026-04-29; corroborated across 5 sessions)
- Legal dispute (June 2026): Elliott engaged in a legal matter involving Bricks and Minifigs — his position: the company deflected responsibility, creating an awful customer experience that snowballed; case notes weigh how opposing counsel would read a Nov 14, 2024 payment-reconciliation exchange (source: hermes_sessions/3b2200ea-37d0-403b-f779-5a51514b3fb7, 2026-06-23)
Migration to eco-chi-001 (June 2026)
- 2026-06-23: Hermes migrated VPS → eco-chi-001 (gateway + mcp-proxy on 8765/8642/8644, nginx cutover, VPS standby); 50+ path fixes; 130 skills (38 recovered from Seagate), 20 crons; the 7–8 hr estimate was wrong — real work ~1 hr (902MB rsync over Tailscale, venv rebuild, 2 systemd units); “all laptops are dumb terminals” rule locked into CLAUDE.md the same day; CLAUDE.md on eco-chi-001 first created 2026-06-23 — before that CC on this machine had zero session protocol; migration pitfall: crons stranded in the legacy profile’s dead jobs.json (Jake + Gary Webb dark during the highest-velocity build week) (source: cc_sessions/1f81898a-a237-baff-888c-ecd658d68f33, 2026-06-23; corroborated across 8 sessions)
- 2026-06-23 queue-hygiene day: eco_queue.md created from scratch (no ecosystem-level task list existed); diverged local/Drive cc_queue.md copies merged, DONE items archived; completion audit found the “done but unmarked” class (CC-001 portal done, CC-003 partial, CC-006 substantially done); research_log.md created on Drive as the standalone central research repository (PARKED entries carry GROK FINDINGS slots; the book pulls FROM research_log, never the reverse); Elliott’s research directive: SuperGrok is the designated primary researcher + second-opinion reviewer (source: cc_sessions/32deacc7-c23b-9c20-ec08-e4d176ee1279, 2026-06-23; corroborated across 6 sessions)
- Session-archive lineage: Session_Archive_work.md (Drive, append-only, Hermes auto-appends via archive_append.py) → Chronicle.md (Gary Webb narrative, C-001–C-058 by 06-30; Drive ID 1rzpoSrXsOysRXZkAb-TcCg56tgXYnxJA) → SMK9_Build_Progress_Book.md (technical/permanent decision record); gap found: CC sessions since 06-18 had no Session_Archive entries (it was Hermes-only territory); cc_sync.py added to push CC session summaries to Firebase so briefing crons see CC work; Chronicle.md (C-001–C-040 era) was once presumed lost but found alive on Drive — the status note simply had the wrong filename; division ratified: Chronicle tells the story, research_log.md holds the raw data; the Book’s original title material lives in the PackRat corpus (94 claude.ai conversations once extracted to VPS /tmp — at risk on reboot) (source: cc_sessions/2a5a7772-ef68-def6-947d-72188374f16b, 2026-06-23; corroborated across 5 sessions)
- Storm outage 2026-06-24/25 exposed the failover gap: eco-chi-001 went offline and Hermes did NOT auto-switch to the VPS (nginx upstream root cause); a later same-week test succeeded — VPS gateway auto-started during a ~10-min hotspot switch and deactivated on recovery; @Ecoverse_Report_bot reports channel live (source: cc_sessions/442f3f67-6d5e-5567-8b74-a4421516bbaa, 2026-06-25; corroborated across 3 sessions)
Cost crisis arc (June–July 2026)
- Jake’s founding sweep (2026-06-24): 10 enabled crons inherited
model: null→ Sonnet by default, ~5–15x unnecessary cost, projected 589 in 14 days); June closed at 873 (11x the 1.58/day) — removed 2026-06-30 (D-030); all 10 LLM crons pinned to claude-haiku-4-5 (LESSON-010); late-June tuning (nudge intervals, kanban 4h net, BRIEFING 22KB→12KB) projected 36/day (source: hermes_sessions/3b69e03b-2036-7066-2983-527f7229938e, 2026-06-24; corroborated across 8 sessions) - June cost incidents: 400+ Opus-OCR leak — root cause: no agent owned model-tier verification (Jake gained a model-tier checklist); July spend shock: one overnight CC session crossed the billing reset ($28.13 in 5 hours, 487 Opus calls, 160M cache-read tokens); a 16h+ overnight Opus CC session pushed ~160M tokens → guardrails: close CC sessions when done, Opus ceiling (source: hermes_sessions/730a0893-2108-71f1-6938-3891661257fb, 2026-06-24; corroborated across 4 sessions)
- Mid-July escalation: 417.56 MTD vs a 0 (source: hermes_sessions/22af032c-5031-ef30-1bfa-feb38c75a68d, 2026-07-16; corroborated across 8 sessions)
- Spend snapshot 2026-07-13 (Jake): July MTD metered Anthropic 137.81 + SuperGrok 479.57/mo total; TASK-CC-COST-001 verified facts (07-18): Hermes routed Anthropic via a METERED key, no Anthropic OAuth exists (add attempt died on HTTP 429 — D-052 ToS carve-out to D-050); Route B (Kimi default) applied 07-18 and verified: first post-Route-B day 307.31/$450 (68%) (source: hermes_sessions/cf19f5c7-c334-f43c-2402-38708a2504d7, 2026-07-13; corroborated across 5 sessions)
- The “56 via Hermes” panics both proved false alarms: Anthropic invoice dates ≠ usage dates; the live config had been switched to claude-fable-5 on Jul 10 and ran ~38h, but 99% of usage was claude-sonnet-4-6 (source: cc_sessions/9d8fdf39-9082-d3b9-df7c-a04ec6fa46f1, 2026-07-12; corroborated across 3 sessions)
Seagate arc (June–July 2026)
- The Seagate 8TB (Backup Plus Hub STEL8000100) held the active smk9-app repo and the Hermes data when it died ~2026-07-02 — it had physically fallen from a shelf shortly before, and emitted smoke during a dock power-up after the enclosure failed; fault isolated to the cable/enclosure USB bridge; fstab
nofailmade mountpoints silently empty dirs on the root fs — services wrote into them unaware (source: cc_sessions/309058cd-856d-183a-0741-8b18cc07e0fe, 2026-07-03; corroborated across 6 sessions) - Loss audit: confirmed lost — CC session transcripts on that path, the Hermes cc-profile Grok OAuth, the personal Samsung phone backup; all code/config had other homes (GitHub, VPS, NVMe); the big April SMS export (9.6GB) was independently safe in Drive; the audit also proved the nightly rclone Drive backup had failed every night since Jun 26 (gdrive remote never configured — no safety net existed); replacement 8TB ordered $212.78; loss assessment CLOSED on the personal side: the laptop→Seagate backup was fully redundant (laptop C: held no unique copies — dangling symlinks to the dead D:, empty stub folders; repo/SMS/Hermes config never uniquely stored there), and the phone’s WhatsApp history and photos live on the phone’s internal storage (the Seagate copy was a redundant Smart Switch mirror) — nothing personal/irreplaceable was actually lost; remaining rebuild items were repo re-clone, Hermes Grok profile re-auth, and the broken rclone gdrive remote (source: cc_sessions/2e752b43-5476-ac38-3dde-74c2144ba46c, 2026-07-04; corroborated across 6 sessions)
Infrastructure sagas (July 2026)
- GitHub Actions saga: Alleycat777/smk9-app never ran a single workflow (0 runs since 2026-07-01); manual dispatch → 422 “Actions has been disabled for this user” — an account-level anti-abuse flag (account created 2026-04-28; burst of new devices/PATs/2FA likely tripped it), not billing (Free plan, $0, minutes available); support ticket #4540585 + a Hermes watch cron (ticket-watching became standing practice); interim deploys via the D-032 clean-worktree method; still blocked through 2026-07-18 (branch 17 ahead, unpushed); an early screenshot diagnosis had wrongly read it as a billing issue (source: cc_sessions/5dedf125-40d5-76d4-8a4c-d374dcff2a11, 2026-07-08; corroborated across 8 sessions)
- Prompt-injection file found on the VPS (2026-07-07): /home/smk9/root_deploy_brief.md tried to trick an agent into reverting functions/index.js to an old snapshot disguised as a normal feature commit, then deleting evidence; created inside a legitimate session window that processed OCR/Gmail content (classic injection vector); no external break-in evidence in auth logs; investigated and removed (source: cc_sessions/1130a1d2-1945-6693-bda9-fc817e944bca, 2026-07-07; corroborated across 2 sessions)
- Grok web-LLM hallucination incident (2026-07-09/10): grok.com web chats claimed to have saved 5 Fable/ICM planning docs and built a tarball — a diskless web chat cannot write files; the “tarball” download was a Google sign-in HTML page; bodies were recovered via Firefox cache forensics + Elliott pasting verbatim, byte-validated; Grok’s own “reprints” were substantially condensed vs originals; canon lessons: web LLM ≠ disk (“I saved it” is hallucination), knowledge locked in chat UIs must be exported to durable storage immediately (feedback_web_llm_no_disk.md) (source: cc_sessions/2b9d2217-d88e-3b27-f198-cea57dc75747, 2026-07-10; corroborated across 8 sessions)
- Fable era: Fable 5 live on Elliott’s Max plan 2026-07-08 (verified via
claude --model fable -p); sandbox at /home/eco/Fable/test-sandbox-icm-fable; The Relay wired the same week (*/5 cron, relay_runner.py + queue.json, Fireman after every [CC] task, [GROK]/[ANY] routing, HIGH-risk escalation gate) and declared ACTIVE/OPERATIONAL (D-043); pre-Fable handoff package was internally inconsistent (5 planning files missing; asserted-live claims like git Time Machine and Voicebox “active” untrue) and was downgraded to a TODO checklist; PRE_FABLE_VALIDATION_PROTOCOL executed end-to-end 2026-07-09 (CC recovery → Grok counter-review → Opus → Elliott gate → Fable), closing APPROVE; Fable_Handoff_2026-07-09 Drive folder holds FABLE_HANDOFF_FINAL.md + a 63-entry tarball; attribution correction: the 07-08 multi-LLM audit was orchestrated by Opus 4.8, not Fable (record corrected) (source: cc_sessions/c170814d-8570-634a-d5dd-09b5272f813e, 2026-07-08; corroborated across 10+ sessions) - Multi-LLM ecosystem audit (2026-07-08/09, Elliott-commissioned, ~20+ verified findings, run on live production): validated the architecture and caught live bugs (Relay [CC] path failing under cron — claude not on PATH); notable findings: the email Gatekeeper existed only as configuration (no code gate at all), Ecosystem_Agents.md overclaimed Relay status, no access logging on PII reads, webhook secret plaintext on 0.0.0.0; remediation tranche: C1 gatekeeper email hook code-gated on the VPS dispatcher, H1/H2 intake-draft auth (deviceToken TOFU), H3 attorney allowlist, H4 QuickBooks gate; laptop-lockout lesson: C2-4 was “FIXED + verified” but verification checked authorized_keys, never that the laptops held matching private keys — verify the full path, not the artifact; log evidence showed the box had been internet-exposed, contradicting the architecture doc (source: cc_sessions/33802a7f-9912-b9c6-c517-655419fa25ba, 2026-07-08; corroborated across 10+ sessions)
- C2 remediation honest tally 2026-07-10 (correction of “fixed them all”): genuinely fixed/live — Gatekeeper email hook, Fireman gate 91/91, GitHub PAT rotated to an SSH-based remote, WhatsApp creds 600-perms, unlabeled VPS root key removed, Firestore daily backup clean, ADMIN_KEY de-hardcoded, certbot timer with all certs valid; NOT fixed — SSH password auth deliberately re-enabled (C2-3 paused by Elliott’s call), Weekly Pulse cron token-free but throwing RuntimeError with no report since 07-05, Firecrawl init failures ongoing, config drift (
approvals.mode: auto; gatewayTimeoutStopSec(90) < drain_timeout(180)), tmux-ecosystem.service FAILED, single-NVMe redundancy unaddressed, DR passphrase-in-Bitwarden unverified (source: cc_sessions/1572ebc0-8483-1b9a-322c-faacc0281278, 2026-07-10) - Grok’s verification of the second-brain/Librarian plan (2026-07-10) surfaced durable findings: the Librarian is hosted by Hermes cron, so if the scheduler/gateway dies the Librarian and its Telegram alerting die too — at least one heartbeat leg must live outside Hermes;
eco_watchdog_001had been disabled since 2026-07-03; Alexandria’sclient_datarouting sends client files tonotes/{project}/clients/, which under the planned git push would put client PII into immutable git history on the Hetzner VPS (exclude*/clients/andpersonal/); the Alexandria cron ran barepython3, which lacks sentence_transformers/qdrant_client (hermes venv required); plus: backups must pass a restore drill (clone-and-byte-compare), the hermesrelay-runneris the scheduler Elliott explicitly approved by name (the crontab*/5was an unaccountable duplicate; the runner’s O_EXCL lock already prevents double-execution), and “dead-letter count = 0” checks must exempt legitimately ESCALATED tasks awaiting human action (source: cc_sessions/17533b73-b8fc-e440-7bb5-a0db8a8f1039, 2026-07-10; corroborated across 2 sessions) - AGENTS.md divergence (/home/eco/AGENTS.md vs repo copy) first flagged 2026-07-07 during CC-019 with the repo copy more current; the same session built session_end_sync.py, which pulls the Hermes session delta via direct SQL against state.db and writes a digest for one consolidated SMK9_Master.md entry (source: cc_sessions/69f13e67-61f5-672d-8e85-efe74c3b1078, 2026-07-07)
- Resynthesis V3.1 (2026-07-10, 3-model panel-amended): reordered to Repair → Connect → Verify → Measure after review found live defects (broken rsync auth, double-scheduled Relay with dead-letter queue, disconnected Alexandria→Qdrant bridge, wiki with zero durability, a PII misroute into notes/); the “Relay is unbuilt” premise was factually false (ACTIVE, 340 scheduler runs) → AMEND verdict; Phase 0 executed: wiki→Qdrant bridge live (wiki_qdrant_upsert.py into the wiki_pages collection), PII misroute quarantined pending a Firestore route (open follow-up), Kobi reconciled into the registry, first-mention recall live back to Apr 25, Fireman 91/91; Phase-0 root causes on record: the second-brain→VPS rsync mirror failed on SSH key selection (no ~/.ssh/config Host entry — fixed so all future SSH consumers inherit it), and the Relay was double-scheduled (user crontab */5 AND hermes cron */10) — reduced to the single Elliott-approved scheduler with 6 dead-letter tasks closed CLOSED_STALE (source: cc_sessions/46b56745-71ef-98ad-931e-71fe383974c1, 2026-07-10; corroborated across 7 sessions)
- Qdrant clobber incident (2026-07-12/13, “Sunday-clobber”): the old weekly qdrant_sync Sunday-2am VPS→local restore leg overwrote pre_hermes_sessions (1,657 full-content points) with an obsolete 387-point VPS snapshot — silently destroying the 07-07 full-history rebuild; the Librarian caught it within 7 minutes; restored same day by re-ingest from the intact source export (
claude_ai_ingest.py --reset, verified 1,657 points, 91/94 conversations) and closed with three-model verification (CC executor, independent adversarial CC verifier CONFIRMED-GOOD 7/7, Grok CLOSED-WITH-FOLLOWUPS); wrinkle on record: the reset had already been executed 2 minutes before the tasked executor ran, from the hermes venv outside any cron — attribution had to be investigated; adversarial verifier flagged residuals: no cryptographic source hash, root-cause fix (commit 975c320) never proven deployed on the live cron; the fix removed the VPS→local restore leg, and pre-audit hardening later reversed qdrant_sync direction to local→VPS, closing the clobber class (source: hermes_sessions/32d0c29d-0fff-41f1-b777-84d0eda5ba1f, 2026-07-13; corroborated across 7 sessions) - Credential-exposure arc (window 2026-06-22→07-12, investigations opened under Grok/Indiana Holmes 07-13): Drive docs found containing an Anthropic ADMIN-level key (can mint API keys) and a second GitHub PAT — Elliott had staged the new PAT in a “Maps Key” Drive doc together with the Maps key and the admin key; the GitHub token printed in plaintext three times (D-046 rotation after CC leaked it into its own transcript twice); credential-hardcoding regressions recurred at 3 independent times (06-22, 07-07, 07-09) by different session instances; all keys moved to local gitignored smk9_keys.env-pattern files and the Drive docs deleted; PAT rotation executed via silent read of ~/.newtok across 6 live locations; a second Indiana Holmes investigation covered VPS git-mirror divergence (source: cc_sessions/1d689055-c60b-0ee7-11af-aff3cd8a03b5, 2026-07-13; corroborated across 8 sessions)
- ADMIN_KEY closure timeline: D-042 open (07-09) → D-045 closed (07-13/18): key fully purged, query-param leak vectors closed, 3 completely unauthenticated bookings endpoints (DELETE/:id, PUT/:id/payment, PATCH/:id/vax-cleared — a 2026-07-07 regression) fixed, dead key scrubbed from all git history, GITHUB_TOKEN rotated — Fireman 106/106 local / 91/91 VPS (source: cc_sessions/da21233b-ddab-ae0e-8e6e-5b36baab551c, 2026-07-13; corroborated across 3 sessions)
- Stable-fileId lineage: cc_queue duplication class (Hermes
drive uploadcreating NEW files instead of updating canonical, 07-02; 20+ duplicates at peak) → fixed canonical fileId protocol → SMK9_Master duplicate incident same week (3 stale copies purged) → SMK9_Master forked again (07-13 ADMIN_KEY branch vs 07-15 D-023 branch), merged with all 30 entries preserved (Exhibit A for session isolation) → D-049 edit-in-place on one canonical fileId; 9 stale SMK9_Master Google-Doc duplicates trashed 07-18 (30-day undo); google_api.py originally had nodrive updateaction — implemented 2026-07-18 (D-047; root cause was a propagation gap: the code existed locally but the installed hermes-agent copy was stale, superset pushed to all 4 installs) (source: hermes_sessions/9f0c2d53-a8f9-999b-6f64-fca650850976, 2026-07-02; corroborated across 10+ sessions) - Decision-ID hygiene: D-023 was reused 3x (booking-lock 06-22, SuperGrok 07-01, ICM-load 07-15) and the smk9-app vs second-brain decisions_log.md copies were unreconciled; the ICM-load decision was reopened for Grok+CC+Elliott consensus and renumbered D-048; queue-drift saga (IL license re-flagged 5+ times) produced LESSON-016 (paused ≠ removed), LESSON-017 (verbal completion ≠ committed), and the Queue Commit Discipline: marking done requires same-turn status edit + git commit/push + report (source: cc_sessions/c3fe8c51-7d22-dc4e-359d-eb5b79b28939, 2026-07-17; corroborated across 5 sessions)
- Agent-gap history: Grandmaster J was marked ACTIVE in ecosystem_graph.json and wired into docs but its SKILL.md never existed (found 06-30, built the week of 07-07 with closes_lesson); three core agents with live crons were absent from ecosystem_graph.json (two sources of truth, no sync); Elmer Fudd, Indiana Holmes, La Senora also lacked SKILL.md at 06-30; agent-da-vinci and agent-relay were “named-but-not-built” at 07-01, both live by mid-July; LESSON-013/014/015 staged (suspended channels need 14-day escalation; every agent needs SKILL.md or archive; critical crons need retry + fallback delivery) (source: hermes_sessions/2856d354-326a-d14f-8d8d-d9f45ed3058a, 2026-06-30; corroborated across 6 sessions)
- Misc mid-July context: T-Mobile home internet cut 2026-07-16 (mini ran on phone hotspot pending AT&T); two power cuts shut the mini down mid-audit 2026-07-18 (work resumed from tmux/task state); week of 07-07→07-14 was the first clean “no fires” week (C-078 heartbeat, 6 capabilities shipped, Fireman 100/100); 2026-07-17 production incident: ALL /api/** routes 404’d starting 00:30 CDT (before that night’s deploy — suspected deployed-version mismatch); Elliott explicitly pinned the debugging to Opus (source: cc_sessions/16433d9b-67c7-5895-2406-78b955fa4039, 2026-07-16; corroborated across 4 sessions)
- The Book: planned long-form human-story compilation of the build (“My Journey with AI — A Journal”, created Jun 11; workshop/lesson-book/course potential), seeded from the PackRat corpus map built 2026-06-11 from Claude memories.json; failures are chapters, not footnotes; audit scope decision: finishing TASK-AUDIT-001 makes the Book current and draft-ABLE (voice model built, 34k-signal sources reconciled, Gary Webb fixed) but does NOT draft it — Book v1 drafting is the first forward-build task after the baseline unless Elliott adds it as a synthesis deliverable (source: hermes_sessions/08ee05f9-257d-9d58-d203-ab26ae39f652, 2026-07-07; corroborated across 4 sessions)
TASK-AUDIT-001 (July 2026)
- Trigger: the ecosystem “silently rotted for a month” — La Senora marked ACTIVE and “confirmed wired in” was silently not doing her job (the “ACTIVE ≠ running” canary), one agent wrote to a dead endpoint 5+ weeks, Spider-Man’s tracker ballooned to 21 open items (append-only file + dormant weekly sweep + Memento flagging with nothing acting), the owner model went 25 days stale; audit origin traces to Elliott’s 2026-07-07 instinct that the ecosystem was built with many mid-flight fixes (source: cc_sessions/be78580a-9c9d-75f4-ea75-925f26250ce4, 2026-07-18; corroborated across 5 sessions)
- Blueprint lineage: scope blueprint at docs/ecosystem-audit-scope-blueprint-2026-07-18.md (authors CC Opus 4.8 + Elliott; executor Fable; independent verifier grades; “the audit fills gaps once, then the nervous system maintains continuously”); hardened through adversarial panel rounds — v1 concerns (“zero UNKNOWN drift” DoD is fantasy → trimmed to 11 items; unverified nervous system; 72h soak), v11 unanimous NO-GO (end-to-end flow proof could touch real bookings/sends), v13-FINAL unanimous 3/3 GO (Grok/Kimi/Opus) with “Dr. Manhattan grades — Fable never self-certifies”; D-051 logged; Kimi elevated to first-class ecosystem member; Grok’s trusting-trust objection ruled out-of-scope (drift, not adversarial compromise); Grok caught a real phase-ordering bug (DM installed in Phase E after the mutations he must grade → moved to Phase A) (source: cc_sessions/4497dac4-45fc-c21b-35ea-10dce4b4996c, 2026-07-18; corroborated across 8 sessions)
- Governance: DM verifier runs as its own OS user
drm(TE-DM-OSUSER tracked exception; Phase C hard-gated on it); hard 15, breach → read-only halt); READ_ONLY.sentinel removal is Elliott-only; ingest crons paused per Phase-D chunk with post-condition PASS resume; audit north star: the ecosystem hoards signal (19k SMS, 9k voice, ~34k vectors) but had only 23 synthesized wiki pages — value is synthesis density, not capture volume; the Elliott self-model reframed as WS0, the center everything serves (source: hermes_sessions/ea1a4ed1-6c7c-6f9a-34c5-843b7a7fe178, 2026-07-18; corroborated across 5 sessions) - Execution: DAG locked at v5/round-5 with unanimous 3/3 GO (every Kimi blocker a genuine ordering bug; DM independently recomputed the pinned spend numbers; discovery: grok-4.20-reasoning runs on SuperGrok OAuth while docs still pointed at grok-4-fast); Phase A certified (freeze holding, 6 jobs paused, 201 items pruned reversibly, 15-min heartbeat, 07-12 clobber class fixed, all 7 Qdrant collections mirrored with matching counts, spend baseline $307.31 MTD); Phase B ran ~10–11 read-only workstreams with every material finding adversarially re-verified by a second agent (findings in 11_ecosystem_audit/findings/); the drmanhattan tmux session died in a reboot and had to be re-stood before Phase C; gate progression 2026-07-18: Dr. Manhattan’s two verdicts landed — drm_isolation PASS (TE-DM-OSUSER closed) and Phase B Discovery Ratification PASS-with-notes (gate B satisfied; notes were downstream escalations) — opening the Phase C gate (source: cc_sessions/9c2aaaaf-d82a-4a96-95e7-d503cd216fc6, 2026-07-18; corroborated across 7 sessions)
- Phase B/C confirmed canonical-doc rot as founding hypothesis: Gary Webb’s Chronicle canonical fileId 404s with 4 Drive forks (two updated 79s apart — his weekly run “succeeds” into a non-canonical copy; its cron prompt uses the banned search-by-name+reupload pattern); Drive decisions_log.md 18 decisions behind the repo while Grok can only see Drive; Layer-0 Core taught wrong priors every session (e.g. Relay “not yet built”); repo AGENTS.md still mandated the fork-generating write pattern; a live substring-collision prune bug had collapsed hermes_sessions Drive retention to 2 of 7 snapshots (fixed on both hosts with env-overridable MAX_PRUNE_DELETES); Tom Skilling rain-watch ran redundantly on VPS + local (source: cc_sessions/11643000-db1b-9cf4-fe19-779866eb9841, 2026-07-18; corroborated across 4 sessions)
- Phase C: 88 verified findings → reconciliation plan (auto-apply only reversible fixes with logged undo paths; destructive changes batched for Elliott as ~/APPROVAL.md + approve.sh, 7 items incl. trashing the 9 stale SMK9_Master duplicates); skill-archive sweep: 82 verified-dead skills archived (incl. a jailbreak skill library) — the verification agent saved 18 skills wrongly called dead and caught default grep silently dropping matches; Phase C closed 2026-07-18 via a DM reversibility ceremony (DM picks fixes, CC undoes, DM verifies through his own read paths, CC re-applies, DM signs) — DM-signed reversibility verdict PASS on 5 full undo→reapply round-trips with all hashes DM-derived, unblocking Phase D; scope arithmetic: Phase B’s 395 scoped items reduced to 88 verified findings; standing advisory: the jailbreak library is one-way by design — restoring it always requires Elliott’s explicit word, considered correct behavior; Phase C also staged ICM corrections (c1_staging, per-edit live-verified): Ecosystem_Core claimed hermes v0.15.1 (live is v0.18.0; Qdrant v1.18.2), the Memento cron was documented “8am CST” but is really 2pm CT (frozen for audit), the “file IDs change every overwrite — never hardcode” rule was inverted for the 5 canonical stable-ID files, La Senora’s false auto-trash claim was replaced with the D-049/D-051 report-only mechanism, and the nonexistent
llm-routerskill was replaced with the real config.yaml routing (source: cc_sessions/71b2424b-cfa0-1c6c-698f-6e8aaa908236, 2026-07-18; corroborated across 5 sessions; also cc_sessions/7e37cc42-6b9d-dfc9-f3ea-d5d0cc6f75c8, 2026-07-18) - Phase D: D-053 ratified with a pinned census as ledger denominator — cc_sessions 2902, hermes_sessions 1533, pre_hermes_sessions 1657, smk9_client_stats 100, smk9_sms 19388, smk9_voice 9309, wiki_pages 23 = 34,912 total; Opus panel flagged PII risk (re-synthesis writing to git-tracked notes/ without 100% scanner coverage) → Phase D PII regime; keystone: an actively-learned Elliott model from ~9k voice + ~2.6k session + ~19k SMS records; D-2 completed 2026-07-18: Elliott Model v1 live — 166 source-tagged claims (voice, decision heuristics with reversibility as master axis, risk posture, failure modes), 12 pattern-inferences quarantined, consumed by Gary Webb, Jake, and every Relay CC dispatch; audit discovery seeds: elliott_personal collection VPS-only, VPS disk 81%, a stale cron, foreign uncommitted changes in ~/.hermes/scripts (source: cc_sessions/885d1e00-beab-f652-e0f6-8d9680e1bbf3, 2026-07-18; corroborated across 6 sessions)
- Cross-agent memory gap proven 2026-07-12: CC→memory ingest ran on a daily 5am cron and Hermes recall searched only the notes/wiki store (never cc_sessions Qdrant) — CC decisions were structurally invisible to Hermes until written into the searched store; compaction pathology on record: a stuck “Giovanna” investigation persisted 60+ compaction cycles with zero new information (source: cc_sessions/23bea785-c321-0d64-bcd3-8daa3e16a919, 2026-07-12; corroborated across 2 sessions)
- Hermes-isolation root cause (2026-07-17): the recurring “session collisions” were NOT a second Claude — the concurrent writer is the always-on Hermes gateway editing /home/eco/Dev/smk9-app directly and committing as Elliott’s own git identity; caught mid-edit autonomously adding debug logging to functions/index.js (BUG-001) — the concrete collision class driving the Hermes↔CC isolation plan; Grok’s adversarial review flagged undefined merge-back, identity-dependent parsers, uncovered shared surfaces (~/.hermes/data, Firestore) and proposed a dedicated POSIX user (source: hermes_sessions/b55d2890-c19e-884c-872a-4796a94d486f, 2026-07-17; corroborated across 3 sessions)
- Standing decisions at slice end (2026-07-18): D-047 RESOLVED (drive update), D-050 (OAuth/subscription-first) + D-051 (La Senora 3-layer janitor, explicitly not to be reverted) STANDING; git origin switched SSH→HTTPS after GitHub rejected id_ed25519_github; Fireman local 106/106 / VPS 91/91; audit blueprint file deliberately left uncommitted per Elliott; 4 drifted hermes skill installs propagated to md5-identical copies; decision-anchor context: D-001–D-009 graduated to Drive 2026-06-09, D-010–D-029 active with rewind paths in early July, SMK9_Build_Progress_Book.md is the permanent record (source: hermes_sessions/da920cde-1426-cf8a-ccd7-3e7288e78e8d, 2026-07-18; corroborated across 4 sessions)
- Historical failure-mode exhibits now cited as doctrine: Voicebox Phase 4C “done in docs, never executed” (doc-vs-system drift); the fabricated-address incident (a cron wrote fake data because web_search/web_extract were dead → Places API + ddgs wired, anti-fabrication rules, circular-citation guard CC-030/031); Twilio rejection email with the exact fix sat unread for days while email access existed (origin of the email watchdog); C2-9: a key assumed dead was LIVE in prod — live-test credentials before declaring them dead; VPS divergence (ran independently ~2 weeks, 25 diverged commits audited, every real fix rescued incl. the 25MB body limit that had silently rejected signed intakes) (source: cc_sessions/e3973c96-ed44-5a27-0030-70d3757b95e0, 2026-07-08; corroborated across 8 sessions)
- Prior self-audit precedents: the “claude-doctor” npm tool reviewed 9 sessions (May 7) diagnosing correction-heavy/negative-drift/error-loop patterns → 5 behavior rules; Van Clief ICM gap analysis ran CC + Grok in parallel tmux sessions (2026-07-02) evaluating 4 proposed ICM fixes against the source paper; ICM compliance finding: the ecosystem implements Layers 0–1 only — an always-on multi-project assistant, not the paper’s staged content pipeline; transferable pieces = Layer 3/4 separation, Inputs-table discipline, edit-surface/portability, Edit-Source Principle (D-032 in decisions_log); the actual Van Clief ICM paper survives durably (Drive PDF + local copy at Dev/smk9-app/_research/2026-07-07-second-brain/van-clief-icm-paper.md); a full read established that ICM is designed for sequential, reviewable pipelines and explicitly does not cover real-time multi-agent collaboration — Ecosystem_Core borrows ICM’s Layer 0–4 terminology but its Layer 2 is ongoing project state, not per-stage Inputs/Process/Outputs contracts; a deliberate category difference, not a bug (source: pre_hermes_sessions/7a4272e5-15eb-828e-c37a-ce2372837361, 2026-05-07; corroborated across 6 sessions; also cc_sessions/8c7e6304-8dc8-f682-a411-b5a65fc0ba7e, 2026-07-07)