tech-build-decisions
Merged D-3.2 wiki page — curated from 13 slices spanning pre_hermes (Apr–Jun 2026), hermes, and cc_sessions (Jun–Jul 2026).
Compute & network architecture
- Long-term compute architecture (decided pre-purchase, May 2026, in force since): mini PC = primary local Hermes host and single compute node; VPS demoted to warm standby/failover (D-018); Firebase = edge/sync only; laptops are thin terminals that SSH into the mini (source: pre_hermes_sessions/bf7ec1f0-20ab-5c65-5309-a04073e41167, 2026-05-07; corroborated across 5+ sessions)
- Mini PC selection: Minisforum UM870 Slim (Ryzen 7 8745H, 32GB DDR5, $665) picked as best value after setting a 32GB minimum for AI work; GMKtec avoided over reported preinstalled-malware findings on Acemagic sub-brands; the crashing i3/8GB Lenovo was the forcing function; earlier rule from mispriced GMKtec quotes: verify listings before purchase claims (source: pre_hermes_sessions/ce414283-9935-a4c3-56c1-15995f36f450, 2026-05-12; corroborated across 3 sessions)
- eco-chi-001 RAM ground truth: single Micron 32GB DDR5-5600 SO-DIMM in Channel A, Channel B empty (single-channel), 64GB max,
28Gi usable; the matching-stick upgrade ($380+) was assessed and DEFERRED for no current ROI (D-037) after Elliott demanded an honest ROI — the initial ROI was overstated and corrected (source: cc_sessions/9a3ed3c8-c1f6-9b02-3c3a-c4954f1c7c36, 2026-07-10; corroborated across 5 sessions) - VPS: Hetzner CX22 (2 vCPU / 4GB / 40GB, ~5 tiers give only 1vCPU/2GB (Hermes needs 4GB) (source: pre_hermes_sessions/38f55186-017b-db75-38c5-abce8c41bdcd, 2026-05-06; corroborated across 4 sessions)
- Post-migration network path: Internet → DuckDNS → VPS nginx (permanent SSL terminator) → Tailscale → eco-chi-001 ports 8765 (mcp-proxy) / 8642 (API) / 8644 (webhook); VPS keeps 127.0.0.1 services warm; mcp-proxy (pipx 0.12.0) bridges stdio hermes-mcp-serve to HTTP/SSE; ~30s auto-failover to VPS standby (source: cc_sessions/08e11001-77db-eb67-0b33-9cd64e8bd9bb, 2026-06-23; corroborated across 3 sessions)
- Tailscale is the zero-config private mesh across laptop/mini/VPS (planned May 2026, confirmed as the remote-access answer); eco-chi-001 LAN IP 192.168.12.12; Windows laptops connect via PuTTY/SSH user
ecokey auth (source: pre_hermes_sessions/39ef58a1-e52e-997a-ae81-1d15cfee7159, 2026-05-01 + hermes_sessions/518a163c-e352-dcee-7ce7-a90ca8743b07, 2026-06-23) - Known hardware constraints: wired NIC enp2s0 unplugged (NO-CARRIER, box on Wi-Fi only — Ethernet recommended for a 24/7 server); historically flaky 8–9am network window (soak tests cover it); mini ran 2.8GiB into swap with ollama/qdrant/gateway coexisting (systemd-oomd active); VPS root disk ~78% of 38GB; single-NVMe storage is a hardware ceiling, not a config gap (source: cc_sessions/af8523d9-3b43-4766-c0a2-d3a4fedf8e37, 2026-07-13; corroborated across 4 sessions)
- Planned separate mini-server (Beelink SER7) cancelled — eco-chi-001 fills the role, dropping ~212.78) (source: cc_sessions/424e724c-5386-1073-af85-a4befb01b133, 2026-07-13)
- Hardware roadmap research (May 2026): five categories — facility cameras, mini PC, Raspberry Pi, smart door lock (client temp codes + staff arrival tracking), camera glasses; UniFi/Ubiquiti flagged high-priority for unifying cameras+door+network under one vendor with local APIs; smart-glasses eval scored 4 candidates, Brilliant Labs Halo can hit a Hermes endpoint directly over HTTPS with no mobile broker (source: pre_hermes_sessions/dfcf968f-85eb-fcf4-6269-b4690a7745a0, 2026-05-07; corroborated across 3 sessions)
- The upstairs ThinkPad terminal ran Windows 7 (6+ years unpatched); accepted in principle: wipe → Ubuntu since its only job is SSH into the mini (source: hermes_sessions/699234e4-1d9a-2ca8-d5f8-c2c807bee54f, 2026-07-13)
Storage: Seagate saga & backups
- Seagate decision (2026-06-23): keep NTFS + fstab persistence (UUID F4F8014CF8010E94 → /mnt/seagate-ntfs, ntfs-3g, nofail) over reformat-to-ext4 — only Docker-on-Seagate (Frigate cameras, Phase 6) truly needs ext4, so conversion was locked to the camera purchase (“don’t bring it up again”); explicit resolution on record: “Seagate ext4 DEFERRED — cameras first, partition second” — the approved partition plan is for camera footage/Docker/VMs, NOT Hermes; Hermes runs off the internal NVMe with no Seagate dependency, so the migration had no hard blocker (source: cc_sessions/a5f056c8-bde1-2806-6122-32677b3adee8, 2026-06-23); after the replacement drive, the mount path changed to
/mnt/seagate/(not/mnt/seagate-ntfs/) (source: cc_sessions/3f37c929-d3a0-07ad-7480-d68df6db577e, 2026-07-10); a dual-model-approved (Claude+Grok) partition plan existed but was mooted by the drive’s death (source: cc_sessions/144f5d43-cc3a-76ab-0550-b7f83b5dc670, 2026-06-23; corroborated across 6 sessions) - Seagate enclosure died early July 2026 — failure isolated to cable/enclosure internals (byte-identical across three PC-side connections); Elliott’s typo’d shutdown commands (‘unmount’, ‘systemcli’) were no-ops and did not cause it (source: cc_sessions/85b674da-8e13-23ac-4015-fcb75f65c4f0, 2026-07-03)
- Backup lesson from the death: the rclone
gdriveremote had never been configured — the 3am Drive backups were silently not running the whole time; post-mortem storage split: NVMe for working data, future external drive for bulk/cold, no full VPS mirror recreation; Docker data-root (silently landing on the root disk) relocated to /home/eco/docker-data with all 5 *arr containers recovered; ollama fixed to run as user eco with models on the NTFS mount (had been crash-looping on an unwritable Docker-overlay path) (source: cc_sessions/43d87821-c27e-28f3-ef3c-26824144d42b, 2026-07-04; corroborated across 5 sessions) - Post-death storage architecture: mini 1TB NVMe primary → VPS mirror failover → Drive redundancy for core files, ICM-aligned (source: cc_sessions/ac4971f5-9f0c-efe4-7a17-e92f7bd3f11d, 2026-07-05)
- Replacement 8TB drive plan: temporary ~300GB NTFS bridge partition + ~7.7TB ext4; back up the Win7 laptop (~200GB) then the Win11 Lenovo over the bridge, verify, delete bridge and grow ext4, then Samba/NFS share and reconnect the *arr stack; Win7 laptop then flashed to Linux (source: cc_sessions/8a7bb4be-d709-7292-9e8d-2c2e3f04958b, 2026-07-15; corroborated across 3 sessions)
- Media architecture (pre-Hermes decision that still governs): WhatsApp rejected as media hub (compresses video, phone-bound); the 8TB drive on the mini is the permanent local archive; pipeline phone → Google Photos → drive for editing and YouTube publishing; mini doubles as video-editing workstation (source: pre_hermes_sessions/0154f349-dfbe-99c3-795c-3caf6db8f0e0, 2026-06-05; corroborated across 3 sessions)
Session & process discipline
- tmux is the standard for anything long-running (>~2 min) or destructive — a bare Bash command dies with the CC session; auto-attach was extended from SSH-only to all interactive terminals with a no-nesting guard, then the .bashrc auto-attach was REMOVED 2026-06-30 (it hijacked every terminal and broke
firebase login:cibrowser flows) in favor of a manualtwork()function; sessions are offered join/kill/skip at session start; a tmux watchdog (systemd oneshot + alert) was live-tested by killing the tmux server; the mini’s 5-session tmux layout (main+cc/grok/hermes-bg/monitor) is Elliott’s standing workspace, not stray jobs — session-start triage should treat these as persistent, not offer to kill them (source: cc_sessions/46516d22-73e3-ff0b-92a5-faad13487446, 2026-07-14); tmux-ecosystem.service failure root cause: its ExecStart pointed at the dead Seagate path/mnt/seagate-ntfs/hermes/scripts/tmux_bootstrap.sh— the live script is/home/eco/.hermes/scripts/tmux_bootstrap.sh(a distinct unit from tmux.service/main); incident class on record: a local terminal emulator window froze/detached visually while itstmux attachon pts/2 kept running server-side continuously since Jul 7 — emulator freeze, not tmux/SSH failure (logged separately at Elliott’s explicit request) (source: cc_sessions/035cdda5-b51b-c36f-30e3-8fbea973603c, 2026-06-23; corroborated across 8 sessions) - PD6 output-durability rule (D-031/D-034): no research/analysis/background task is complete while its only copy lives in /tmp, RAM, or an unsynced buffer — output lands in git or Drive first. Born from the 2026-07-02 Van Clief comparison loss (/tmp is tmpfs; a reboot erased both output files); enforced by advisory PreToolUse hook
~/.claude/hooks/pd6-durability-check.sh(source: cc_sessions/2d97e958-b3e9-93fa-3b14-77ad96f09dc5, 2026-07-07; corroborated across 3 sessions) - Decisions-log numbering collision between concurrent sessions (two D-031/D-032 histories) resolved by renumbering to D-034/D-035 — evidence for stable-ID, merge-before-write discipline (source: cc_sessions/1702b102-7927-8257-ee66-6cd26983256d, 2026-07-07)
- Concurrent-session guardrails (2026-07-07): claim markers in cc_queue.md, never-deploy-locally rule written into cc_queue.md/CLAUDE.md/AGENTS.md;
whoscodinghelper at /home/eco/bin shows every live Claude proc, Hermes LSP editing, Relay worktrees, repo dirty state; root cause of “ghost” collisions: Hermes commits asElliott Soto <elliottjsoto@gmail.com>— same git identity as Elliott and CC (source: cc_sessions/f0d57f6b-18ce-99f5-40db-94f26242b623, 2026-07-07; corroborated across 4 sessions) - Hermes git-isolation plan: Grok REJECTed the worktree approach (shared .git still contends), recommending a separate clone with local bare repo or a dedicated POSIX hermes user with chmod-750 checkout; the actual fork generator was session_end_sync.py-era “create a NEW version each session-end” — fixed to edit-in-place on one canonical fileId; deploy-sync gap class: a VPS-local commit (65b8e03) was never pushed and got orphaned by a CC history rewrite (source: cc_sessions/0814cb0d-b455-4b34-6076-c039caada7c1, 2026-07-17; corroborated across 3 sessions)
- Shared-repo hardening (2026-07-17): Drive→local sync crons must refuse to overwrite locally-modified files newer than the Drive copy (dirty-guard — a sync previously ate TASK-CC-COST-001); snapshot crons write to a separate _snapshots/ folder (source: hermes_sessions/4a294eb8-5444-8aa6-7b90-d0cfe12988e4, 2026-07-17)
- Repo policy: a topical folder gets its own git repo only when it holds code that changes over time; documents belong on Drive. Fable work lives in a standalone repo at /home/eco/Fable — deliberately NOT in smk9-app, because smk9-app CI deploys on any push to master (source: cc_sessions/7eb34139-2de9-21ba-0711-3b22a2f12d0f, 2026-07-07 + b4a5b10c-78d8-e7b1-7f54-428376d39300, 2026-07-10)
- Edit-Source Principle (adopted 2026-07-07, Van Clief & McDermott arXiv:2603.16021 §6.3): a recurring correction means edit the SOURCE (stage contract / reference / standing rule), not the output (source: hermes_sessions/aa482d2f-9375-13be-db20-1c7325a36e4b, 2026-07-08)
- Permission principle: an agent may not widen its own permission rules; settings.json grants are human-only. Related: don’t repurpose service credentials — Hermes’s GITHUB_TOKEN shouldn’t authenticate CC’s git pushes; a dedicated GitHub SSH keypair (~/.ssh/id_ed25519_github, 600) keeps GitHub access independently revocable (source: cc_sessions/98bce02d-e236-ba0c-ded3-e3e86fa83dd0, 2026-07-13; corroborated across 3 sessions)
- Investigation protocol: load session history first (the cause is usually in the transcript), check live vs source state, never diagnose without evidence, “leave a Polaroid” — a structured artifact the next session picks up without re-explaining (source: hermes_sessions/44579ede-4a88-045e-47c2-35bfc00b7c3d, 2026-06-26)
- The Claude Code auto-mode permission classifier cannot see conversational permission grants — a denial stands even after Elliott’s explicit go-ahead in chat; only a permission rule in settings changes it; working pattern: hand Elliott single ready-to-run
!lines (source: cc_sessions/93470e11-fd52-0ca9-db69-7acb92620c5d, 2026-07-18) - Claude Code loads CLAUDE.md files by walking UP the directory tree from the session root — rooting in
/home/eco/Dev/smk9-apppicks up both the project CLAUDE.md/AGENTS.md and the machine-wide/home/eco/CLAUDE.md; project-rooted sessions endorsed over the historical Desktop root for repo work (restores git-aware features like /ultraplan, kills absolute-path drift; Desktop was originally chosen as a neutral multi-project root) (source: cc_sessions/583d5000-b62b-3626-c3eb-7bd6dae73893, 2026-07-07) - Watchdog design principles: observe-only (never mutate state — the old version auto-bootstrapped tmux sessions) and silence-when-green; eco_watchdog.py is a free no-agent cron (disk, services, mounts, memory, security updates, tmux, tailscale) with ESM-noise filter, heartbeat moved off the dead Seagate to ~/.hermes/data/, reboot-required-only update alerts; monitored
SERVICES = ["hermes-gateway", "hermes-mcp-proxy"]with tailscaled + ssh approved as additions; runs via Hermesno_agentcron (jobs.json; script stdout piped straight to Telegram, no LLM); standing decision from the same build: terminal/session continuity is a hard requirement — boot-time tmux persistence plus every new SSH terminal session auto-attaching into tmux; alert paths were validated with REAL end-to-end firing (not mocks) at Elliott’s insistence (source: cc_sessions/e0d53f2c-723f-6b64-f77e-bd0c2bcaa6ef, 2026-07-10; corroborated across 6 sessions)
Deploy & validation (SMK9 app)
- Never deploy from local state:
firebase deployships whatever is on disk — a redeploy for an ADMIN_KEY refresh would have silently shipped a held, unreviewed commit (26d451e); D-032 protocol: deploys go via a cleangit worktreecheckout at the exact pushed commit (works even with CI inert); gotcha — fresh worktrees have no functions/node_modules, so predeploy lint falls back to ancient system eslint: install deps first (source: cc_sessions/fc4188c1-e9b9-a410-87ce-0f2884171875, 2026-07-05; corroborated across 4 sessions) - Fireman (~88–106 API/server checks) + Playwright (4 browser checks) live in the repo under tests/ with
tests/validate.sh(commit 0c28e15, —smoke/—browser) as single entry point; tests belong in the repo, not VPS-only; Playwright covers UI/JS the API checks can’t; trajectory 81/88 → 88/88 → 100+; fireman.py made path-portable for eco-chi-001 (source: cc_sessions/43946964-fd0a-b1e2-fd9d-2b7f9687d83c, 2026-06-24; corroborated across 6 sessions) - Fireman caveats:
curl_post()returns(stdout, curl_exit_code)NOT the HTTP status; when Fireman goes red after an intentional contract change, first check whether tests encode the pre-fix contract; two Fireman copies diverged (canonical /home/smk9/fireman.py staler than repo, still hardcoding ADMIN_KEY; prod test-draft cleanup leaving PII test docs); scope: the Fireman/deploy checklist applies to the main app only, not standalone cron scripts — a refinement of the CORE rule confirmed in practice on a Kobi-watchdog fix (cron-script-only change, Fireman legitimately skipped); that fix’s root cause for the record: kobi_watchdog.py displayed raw UTC timestamps from Firestore/Cloud Logging without local-timezone conversion, causing “wrong times” confusion (source: cc_sessions/ffc24999-3552-ee26-3a89-ff8fe21b857a, 2026-06-24; corroborated across 5 sessions; also hermes_sessions/119e8fc3-994b-bcf9-7066-c731b511d5b8, 2026-07-15) - Firebase deploy auth lessons:
firebase logincan’t run non-interactively in CC; stale FIREBASE_TOKEN env poisoned auth (fix: unset + logout +login --no-localhost); alogin:citoken now lives in smk9_keys.env for autonomous deploys; deploys need SA roles run admin (Gen2), artifactregistry.admin, storage.admin, iam.serviceAccountUser, firebase.admin (+ Cloud Functions Admin, Cloud Build Editor added predictively) and the Cloud Billing API enabled; gcloud ADC proved scope-insufficient — use the firebase-adminsdk SA key (source: cc_sessions/f4f372cc-9c12-5a58-dd7b-39999b35411b, 2026-06-23; corroborated across 5 sessions) - Firebase Cloud Run intermittently serves stale/no function versions (Fireman drops to 81/88 with 404 routes); remedy:
nvm use 20 && firebase deploy --only functions;firebase functions:logreturns empty bodies on 2nd-gen functions — use the Cloud Logging API; API-404 incident 2026-07-17: Hosting /api/** rewrite 404’d on known routes with local HEAD 49e03dd vs deployed ad05ed9 (source: cc_sessions/7b0b05d6-b929-5041-559d-2cbf589453eb, 2026-06-25; corroborated across 4 sessions) - CI deploy (deploy.yml) needs a FIREBASE_SERVICE_ACCOUNT repo secret (only VPS_SSH_KEY existed); workflow_dispatch manual trigger added; replacement GitHub PATs must include the
workflowscope or CI file changes fail; CI correction of record (2026-07-08): with Actions reportingenabled:trueand a repo+workflow-scoped token, the “Deploy to Firebase” workflow had never fired once (all-time run count 0) — the documented “CI auto-deploys on push to master” pipeline had never actually run, so merged security fixes sat undeployed; plan: set the secret +workflow_dispatchtest on reinstatement, manual worktree deploy + Fireman until then (source: cc_sessions/d46fe313-b23e-dfd7-3c9f-57ca992dc94c, 2026-07-08; corroborated across 3 sessions) - Firebase functions dotenv deploys MERGE rather than replace env config — a locally-thin
.env.productionwill not wipe live vars (TELEGRAM_BOT_TOKEN/RESEND_API_KEY confirmed intact via the Cloud Functions v2 API after exactly such a deploy); Fireman’s “TELEGRAM_BOT_TOKEN present” check greps the local repo file, not the live function env — a known check gap queued as TASK-CC-028;notifyElliottwarn-and-skips when the token is unset; the WhatsApp webhook deployed fail-closed (403) pending Twilio creds (source: cc_sessions/348ae1bf-86fe-c5b8-8ce6-52d234266ff2, 2026-07-07) - nginx serves the site from /var/www/soundmindk9/ (an early sync went to /home/smk9/smk9-app/public/ by mistake); rsync repo public/ → webroot excluding
*_intake.html; two index.html files exist (Firebase app vs VPS marketing page) — vps-webroot/index.html is now version-controlled with the manual scp path documented (source: cc_sessions/f4f9c10d-d242-c922-574c-952d39356999, 2026-06-24; corroborated across 3 sessions) - Firestore composite indexes can be created via REST (SA token + collectionGroups/indexes) without the Console; shipped-but-never-worked lesson: every GET /ops-tasks query failed in production for 13 days due to missing composite indexes — Richie’s task flow never actually worked despite API+skill fully built; fix = 3 indexes in firestore.indexes.json (source: cc_sessions/4a51d5ad-5e11-2500-bf8b-2a7d2b037db4, 2026-06-24 + e9f8ea6b-151a-ca23-dc13-0d7dcf7366b1, 2026-07-07)
- Broken-web-feature checklist:
node --checkthe extracted JS, read the browser console error first, diff live deployed file vs git HEAD (they may not match), check new string literals for unescaped apostrophes, verify file-input accept/multiple attributes (source: hermes_sessions/11092d08-de45-d071-79ae-ec79950ad1ef, 2026-06-25) - Vax-upload 403 bug class: the multipart fallback path (Safari/iPhone when Firebase Storage JS isn’t ready) sanitizes phone-based clientId (“+1…” → “_1…”) while the server guard (functions/index.js:473) compared the raw value against
vax/{clientId}/— path-validation mismatch; fixed via decodeURIComponent + preserve ’+’ in pathPrefix (commit 76177c2), deployed 2026-07-17/18 (source: hermes_sessions/391076db-b84c-599e-a8a4-f661c3054c1e, 2026-07-15; corroborated across 4 sessions) - Paper intake pipeline: POST /staff/process-paper OCRs multi-page paper forms via Claude vision with HEIC→JPEG; uploads go straight to GCS (memoryStorage/busboy
parseMultipart(), multer removed — Cloud Run 2nd Gen pre-buffers req.rawBody; 10/20MB caps), never local disk (source: hermes_sessions/629839b1-391d-d116-291e-b0b20c80b719, 2026-07-03; corroborated across 2 sessions) - Intake draft endpoints use bind-on-first-write draft tokens: wrong/no token → 403 draft_auth_failed; GET /draft requires the admin key; verified with an 8-case matrix (source: cc_sessions/c568f8cb-016d-bd64-5ef2-6502ed6aecca, 2026-07-08)
- Security-fix ledger (beyond the 06-24 audit): commit ea97a80 had removed the adminKey gate from admin endpoints (restored);
PUT /:id/payment+PATCH /:id/vax-clearedhad no auth (now X-Admin-Key + Fireman regressions); DELETE /bookings/:id was completely unauthenticated in production (gated); hardcoded ADMIN_KEY literal found in tests/fireman.py → rotate + read from env; old key still in git history (fedac54) — filter-repo scrub deferred until no concurrent session is committing; standing lesson: verify secrets against live prod, not prose — an “assumed dead” key was still accepted (source: cc_sessions/9bdab75e-6b33-a127-0cb0-2a8018269983, 2026-07-07; corroborated across 6 sessions) - Emulator gotcha for this repo: functions declare
defineSecretsecrets (ADMIN_KEY, CALENDAR_*), so emulator runs need afunctions/.secret.localwith dummy values — otherwise Secret Manager 403s cascade into misleading test failures (source: cc_sessions/6842b062-e396-dee1-3791-4f2e0869983b, 2026-07-08) - Intake bug class: some
clientsdocs were created under random UUIDs instead of phone-number doc ids; Kobi’s stuck-form checker did raw REST GETs where ’+’ in doc paths must be URL-encoded %2B (Admin SDK handles it, the Python REST port didn’t) — false “stuck form” flags (source: hermes_sessions/e0960f6d-c23f-8215-e822-d576ead0994d, 2026-07-07) - CC-015: all Firestore writes forced to Chicago (CDT/CST) timestamps, not UTC; staff dashboard is read-only for bookings (no creation UI) and HOLIDAY_RANGES is hardcoded only through 2027-01-04 (source: hermes_sessions/aef3abae-f8f2-317a-8bb2-9da4a564d4fe, 2026-07-07 + 5ef71c2b-a131-cb27-6a15-071489b24763, 2026-07-03)
- Firestore backup: daily 4am free no-agent cron exports to Drive, retention 14 days (client-data loss window 7d → ≤1d, closes D-011); backup_firestore.py stringifies Timestamps/refs (json default=str) — restore-fidelity caveat verified via emulator round-trip (source: cc_sessions/da000de1-70a8-4e3d-98b9-10d230fb892f, 2026-07-09; corroborated across 3 sessions)
Cost control & model routing
- Anthropic spend incident (June 2026): real usage 1,005 incl. 15% Chicago tax on ~201 Stripe ~496 was 132.3M 5-min cache-WRITE tokens at $3.75/M, missed because the estimator read flat cache_creation_input_tokens (0) and not nested cache_creation.ephemeral_5m_input_tokens; root cause: long-lived Hermes Telegram sessions dragging ~100K prefixes re-reading ~900M cached tokens; Fable 5 (wrongly set as default) was fully exonerated (2 in / 41 out tokens); model choice was never the cause (source: cc_sessions/19839c31-97b0-95ca-da5d-bc39da50c079, 2026-07-12; corroborated across 8+ sessions)
- Spend fixes: cache_ttl 5m→1h (mirrored to VPS); Telegram session idle-reset >24h; Hermes compaction threshold 0.5→0.08 (protect_first_n 3, protect_last_n 20) so the compressor actually fires; recall-on-demand wired into pre_llm_call.py via a memory-query detector (additive, never blocks) pulling from local Qdrant at 450 this month / 450) (source: cc_sessions/0ca13dd6-3ae7-6cef-3110-1701a0fff6b1, 2026-07-12; corroborated across 8+ sessions)
- The Second Brain was half-built: REMEMBER (store+ingest) worked but FORGET (shrink live prompt, recall on demand) was never wired — paying for both halves, getting neither; ICM layers only shrink the knowledge/system-prompt side and never governed conversation history — the compactor + idle resets were the missing half; corollary finding: “end each session per task” never saved money because it targeted flat-rate Claude Code, not the metered API; expected settle 1k (source: cc_sessions/e046b8d9-284d-95b6-7e66-15493edeb9e3, 2026-07-12; corroborated across 4 sessions)
- D-022: a runaway interactive Telegram session burned ~10.4M cache tokens/24h on claude-sonnet-5; interim fix swapped Hermes default to grok-4-fast/xai-oauth; the referenced follow-up TASK-CC-COST-001 was never actually created (phantom task) (source: cc_sessions/530be09c-da60-3ccd-1d45-8b82a9ca4026, 2026-07-15)
- D-050 (“subscription before metered”, origin: Elliott’s standing rule 2026-07-17): research verdict — only SuperGrok is unambiguously flat (300 Heavy) but xAI’s consumer ToS bans automated/non-human access (least ToS-safe for an autonomous gateway); metered Kimi/OpenAI are ToS-clean; Hermes had been burning ~20 Plus subscription is message-capped (5-hr window) with OpenAI explicitly directing automation to a metered API key — Grok is the only genuinely flat option, and the full report existed only in /tmp at capture time (source: cc_sessions/52c16cdc-5521-ff96-44c0-e7eed8db501e, 2026-07-18; corroborated across 6 sessions; also cc_sessions/1f8dd3ba-1d8a-be18-d8e7-28b9c5f2933c, 2026-07-18)
- Route B applied 2026-07-18: Hermes gateway default → kimi-k2.7-code (provider kimi-coding, fallback kimi-k2.6), functionally verified — routes Anthropic-class traffic onto already-working subscriptions with zero contention with Elliott’s interactive Max quota and no new 429/ToS surface; marked “applied — NOT closed” pending spend confirmation;
hermes auth add anthropic --type oauthis what stops fallback to the metered Claude API key; residual claude-haiku-4-5 pins in add-alert/close-alert/sal-the-plumber deliberately kept as a built-in “automation cost” meter with its own line in the daily 18:00 CT spend report; open watch items: an unidentified sonnet-4-6 pin (~$1.69/day) and Kimi credit-pool drain; gateway default verified at 2026-07-18 session close:kimi-k2.7-codeon providerkimi-coding; the Route-B research report was persisted from tmpfs to git asdocs/cost-routeB-llm-research-2026-07-17.json(PD6 in action) (source: cc_sessions/3736e5ea-ebba-a1fd-e541-06bbdf8d162f, 2026-07-18; corroborated across 10+ sessions) - AI subscription stack (all billed via Google Play under “Sound Mind”): Claude Max 137.81 w/tax), SuperGrok 33.06 w/tax, renews the 23rd), Google AI Plus (2TB Google One + Gemini) 114.99 w/tax); no OpenAI subscription exists; Anthropic API spend is metered separately from the subscriptions (source: cc_sessions/3c762dc8-9ea0-bb99-d068-4123506854b0, 2026-07-13); standing cost flags: sonnet-5 intro pricing ends Sept 1 → Hermes API cost rises ~50% if still routed there; parked from the voice work: the Grok
--yolostanding-permission rule still needs adding to settings.json (source: cc_sessions/46516d22-73e3-ff0b-92a5-faad13487446, 2026-07-14) - Verification rule: model self-reports of identity/routing are unreliable — endpoint configuration is ground truth (a Kimi endpoint’s model misidentified itself; “narrated-done” failure class) (source: cc_sessions/0e0d0f7a-f2f3-016a-9e85-3ed1aa5f7c58, 2026-07-18; corroborated across 3 sessions)
- Cron cost rule (canonized at three levels — live conversion, CC memory, Ecosystem_Core Layer 0): crons default to free
--no-agentscripts; tokenized agents need justification + approval and their mode disclosed up front; free automations get a standing yes; sweep outcome: 4 of 6 agent crons converted to $0 (Daily Briefing, R&R Glass, Memento, Spider-Man); Jake and Gary Webb stayed on Claude after local-model tests failed (a local llama trial on daily-briefing returned SILENT and was reverted) (source: cc_sessions/16dbff07-2f38-b22c-231d-9df973ccb8a7, 2026-07-07; corroborated across 8 sessions) - Kleinfeld cost-auditor design:
no_agentweekly jobs.json audit + one Haiku pass, flags zombie crons and convertible LLM crons; universal model-tier rule: every task declares its minimum sufficient tier — one tier up needs justification, two tiers up needs an explicit ROI estimate (all providers) (source: hermes_sessions/de6d94a3-37e6-a1fa-e88b-fc09e35246b3, 2026-06-25) - Model dispatch doctrine: subscription-first; single best model per sub-task; deliberately swappable top orchestrator (Fable 5 trial → Opus picks up the contracts unchanged post-2026-07-12); high-reasoning planning → Fable 5/Opus, research/adversarial second opinion → Grok; Kimi (pay-per-token) fallback for heavy parallel research; local LLM for private/offline/cheap bulk; on the Hermes bus
--providerREQUIRES-m; never hand execution to a web-chat LLM — it hallucinates success/file saves (2026-07-09 lesson); cost mode declared before every dispatch; only CC and Hermes touch disk (source: hermes_sessions/277bcdab-12c8-aabe-11d9-45e4c60afe3c, 2026-07-10; corroborated across 5 sessions) - Pre-Hermes routing lineage: Sonnet default / Opus reserved for architecture (“C-suite”) set 2026-05-01; direct Anthropic API chosen over OpenRouter/Nous Portal — prompt caching saves ~83% on the per-message voice-profile context (“LLM = Sonnet 4.6 direct, never OpenRouter” locked in the Decisions Log); claude.ai heavy MCP-tool sessions are the most expensive lane (no caching, base64 round-trips) — heavy rewrites route to Claude Code on Max, planning stays in claude.ai; Hermes API calls always burn API credits while CC draws from Max — costs never transfer; June 2 billing spike traced to building an intake form through Hermes’ API key (lane-violation origin of the escalation rule) (source: pre_hermes_sessions/eb66a814-98be-40c6-4213-cf65f1b338a0, 2026-05-01; corroborated across 8 sessions)
Local LLM & voice stack
- Local LLM bar: interactive models must clear 15 tok/s on the Radeon 780M — qwen3:8b 7.5 (fail), qwen3:4b 14.1 (edge), qwen3:1.7b 29.1 (pass), llama3.2:3b 17.28 (chosen primary, D-036, qwen3:1.7b fast-fallback); ollama was silently CPU-only until OLLAMA_IGPU_ENABLE=1 (SuperGrok’s 18–28 tok/s estimate wrong); Vulkan-only — ROCm/HIP is NOT usable on the Radeon 780M (gfx1103): ollama drops it (“no rocblas support”; supported list gfx1030/1100/1101/1102/1150/1151/1200/1201/908/90a/942/950), while Vulkan (Mesa RADV) works but ollama disables the iGPU by default — captured in the idempotent
/home/eco/hermes-setup/ubuntu-bootstrap.sh(source: cc_sessions/0b062b14-8ff6-cc5d-59ef-72000ea48e9e, 2026-07-07); durable system config: models live atOLLAMA_MODELS=/home/eco/ollama-models(NOT the default ~/.ollama, which is empty and has misled searches), with OLLAMA_IGPU_ENABLE=1 and OLLAMA_FLASH_ATTENTION=1 in the systemd unit; inventory incl. qwen3:8b and deepseek-r1:7b pulled 2026-07-07 (source: cc_sessions/896e73c0-c711-f7e9-8ed1-a1a73c52f729, 2026-07-10); Kimi excluded (no small distill; full K2.6 ~1T-param MoE); agent-scale prompts hit context walls (Hermes 64K policy, Ollama default 4096) so local models were never validly capability-tested for Jake-class work; ~10 models installed but none wired into Hermes as a dispatchable provider; ollama formalized as default local provider, unloading models immediately after use (source: cc_sessions/01a2a31d-a9a4-ee5e-8a47-a245fc36141a, 2026-07-07; corroborated across 10+ sessions) - Qwen 3 edges Gemma 4 at similar sizes for reasoning/coding/instruction-following (better interactive default); Radeon 780M shortlist: Qwen3.5-9B/4B, DeepSeek-R1 distills, Gemma 4B, Phi-4-mini, Llama 3.2 3B (source: hermes_sessions/541df4c9-dd56-d630-8a41-aa0e3ba7c29e, 2026-07-07; corroborated across 2 sessions)
- Voice design decisions: no wake-word system is reliable in noise — push-to-talk reusing the keyboard mic button is the fix; the Nordic 1915:1025 USB keyboard exposes its mic button via evdev (hold-to-talk); on the Topippro BL keyboard the mic button reports KEY_PLAYPAUSE (code 164) with a root listener toggling tap-to-record dictation; the keyboard also has a built-in pinhole mic (audio source “USB Composite Device”) that is hardware push-to-talk — it only streams audio while the mic button is held, so dictation can work with no Bluetooth at all (source: cc_sessions/69b931dc-1d10-7971-5972-bd133c03231c, 2026-07-14); Ctrl+Alt+D rejected (GNOME collision, not one-button); ydotool (uinput) is the only reliable key injector on GNOME Wayland (source: cc_sessions/1c6dbf62-277c-f17c-5af5-6f4ff18e9e40, 2026-07-14; corroborated across 5 sessions)
- Jarvis latency story: voicebox’s own /speak orchestration added ~124s overhead per request (TTS engine + hardware were fine) → Kokoro TTS sidecar bypass (~5.6s); keeping the ollama model resident + a reply-length cap got ~2–4s replies; perceived 15s latency was TTS reading ~40-word butler-speak — keep spoken replies to one short sentence; instant wake chime added before the slow local-LLM compose; conversation mode keeps a ~10s follow-up window with no wake word (source: cc_sessions/d364b7a2-5309-dbd8-595b-8210c04d1342, 2026-07-13; corroborated across 5 sessions)
- Dictation hardening after the dead-mic hallucination incident: refuse to record without the configured mic; refuse to transcribe unless ≥0.30s audio with peak ≥ −40 dBFS; block loops via Whisper compression-ratio >2.4; hard 800-char cap; bare “You”/”.” = classic silence hallucination (BT mic disconnected, not a broken pipeline); the temperature-fallback “guard” made transcription non-deterministic and was reverted; loop guards verified live with no false positives on legitimately repetitive speech (source: cc_sessions/2f5c258b-f361-1edf-e304-5298e2396da0, 2026-07-15; corroborated across 5 sessions)
- Jarvis config hardening: dictation hold threshold 2s; wake threshold median+1500 ceiling (startup ambient calibration twice “deafened” it); mic keepalive promoted to a real enabled service; dictate.py refuses to type when no speech detected (fixes runaway dictation loop); documented in ICM Config §12; voice-triggered actions gated by Gatekeeper discipline — HFP mis-transcriptions are common, so a misheard command must never fire an email/action (explicit allowlist + approval line); live-fire debug findings (2026-07-13): the mic keepalive fix confirmed working (strong RMS, clean chunks, zero empty captures), the narrowband HFP mic makes wake-word STT hit-or-miss unless Elliott enunciates, and a designed-in UX gap — “Jarvis,
” enters answer() with NO spoken acknowledgment (only bare “Jarvis” gets “Yes?”), so wakes feel like silent failures (source: cc_sessions/51caa75e-dcb7-9d20-e123-c9bb5b839554, 2026-07-14; corroborated across 5 sessions; also cc_sessions/f504c333-00f6-1552-4695-f45dd6b4a606, 2026-07-13) - Workarounds ratified as canon rather than bugs to fix (audit ruling 2026-07-18): the voicebox torch reload bug (worked around via ollama per D-036) and the PipeWire default-sink flakiness (by-name sink targeting) are tracked exceptions, not open defects (source: cc_sessions/6891626c-d19f-64cf-9169-4b2ad558079a, 2026-07-18)
- Bluetooth voice ceiling: classic BT HFP drops to mono/narrowband when the mic is active (the Sudio A2 problem) — a different earpiece won’t fix it; LE Audio/LC3 on Linux still experimental (BlueZ 5.8x + PipeWire + ISO sockets, not plug-and-play in 2026); boom-style mono HFP headsets are the reliable dictation path; Cleer ARC 5 (open-ear, LC3) is the standout replacement candidate — choosing it would justify enabling LE Audio; Sudio earbuds sometimes pair BLE-only (no audio profile) — remove and re-pair as bonded Classic (source: cc_sessions/51d8c550-fb28-8067-c0f1-d40cb8f0afde, 2026-07-14; corroborated across 4 sessions)
- Mini Wi-Fi/BT: the original MT7902 has no Linux BT firmware (upstream gap) — AX210 was the fix; AX210 tuning: regdomain 00→US unlocked 5/6 GHz, disabling Wi-Fi power save eliminated ~544ms upload jitter (the real streaming bottleneck was Wi-Fi config, not the VPN); voicebox venv must be Python 3.11 (system 3.14 untested for ML deps); voicebox-backend runs as systemd service (health on 127.0.0.1:17493) — it once leaked to 7.4GB RSS, durable guard = periodic restart or systemd MemoryMax (source: cc_sessions/128c6269-cc56-bc01-bbab-1636463a4ceb, 2026-07-08; corroborated across 5 sessions)
Second Brain / memory architecture
- Hybrid Memory Architecture: Layer A compiled wiki/notes (Karpathy-style write-time), Layer B raw vector store (Qdrant + metadata — the only production-grade layer, ~4.1k vectors with live recall hook), Layer C graph/temporal (typed links, Time Machine, first-mention recall); raw→synthesis→compiled flow; provenance (source+timestamp+agent id) non-negotiable; Layer-3 promotion requires Grandmaster J 3-move + Spider-Man blast-radius + human gate; DB contradiction resolved: JSONL + existing Qdrant, no new Postgres/Supabase on a single mini (source: cc_sessions/068faacf-27d2-afc3-91b3-f6fdbd5f45e4, 2026-07-10; corroborated across 8+ sessions)
- Fable orchestrator mission: voice-first, graph-native, agentic Second Brain + AI OS (JARVIS-style); subscription-first; deliberately swappable top orchestrator; Grok first-class secondary for research/audit lanes; build priority Relay → Graph → Voice → Proactive → Real tools; adopted external-review principles: repair before build, connect before expand, a verifier per subsystem (generalize the Fireman pattern), human gate on T2+ crons; reviewer verdict: fix the wiki→Qdrant bridge FIRST — durability layers on an unused surface are polish (source: cc_sessions/031eddc0-49f5-7078-5999-e9c28b53c1ac, 2026-07-10; corroborated across 6 sessions)
- Ground-truth audit findings (2026-07-10): notes/ wasn’t git-tracked; Alexandria→Qdrant bridge was stubbed (writes .md, embeds nothing); the same domain-keyword classifier duplicated verbatim in 4 ingest scripts — deduped into a shared
/home/eco/.hermes/scripts/domain_classifier.pycarrying TWO deliberately distinct keyword sets because the copies were NOT identical (DOMAIN_KEYWORDSbase for cc_ingest/hermes_ingest,DOMAIN_KEYWORDS_EXTENDEDfor claude_ai_ingest adding PERSONAL_HEALTH + FINANCE domains), each caller’s exact behavior preserved, now imported by all 4 ingest scripts + cc_recall’slabel_of; the same build addedcc_recall --first-mention(defensive ts parse + exhaustive scroll over all 4 collections incl. wiki_pages) (source: cc_sessions/61daed49-0b37-0eae-d667-28f6a526f9ba, 2026-07-10); Relay double-scheduled (crontab */5 + hermes cron */10) with a dead-letter queue; sessions rsync failing every run (missing-e ssh -ikey); wiki only ~14–18 pages with VPS mirror covering only sessions/; fixes: single Relay scheduler, newwiki_pagesQdrant collection fed by Alexandria upserts + mtime scan, notes/ git + VPS bare-repo push, time_machine_commit.sh daily commit (5:57am) + */15 sessions rsync (source: cc_sessions/1c9f4e10-a79a-fd16-1c13-2bc46bef03c9, 2026-07-10; corroborated across 6 sessions) - Qdrant: live on localhost:6333; VPS collection inventory 2026-07-18: cc_sessions 2689, hermes_sessions 1415, pre_hermes_sessions 1657, smk9_sms 19388, smk9_voice 9309, smk9_client_stats 100, wiki_pages 23, elliott_personal 3 (elliott_personal is VPS-only, possibly sole copy — cleanup must never touch it); local inventory same day: cc_sessions 2,681 / hermes_sessions 1,413 / pre_hermes_sessions 1,657 / smk9_sms 19,388 / smk9_voice 9,309 / smk9_client_stats 100 / wiki_pages 23, with the VPS mirror having held a stale 387-point pre_hermes_sessions; the “Qdrant VPS→Seagate Weekly Sync” is a
no_agentcron (qdrant_sync.py,0 2 * * 0) (source: cc_sessions/2312c255-e136-5f99-dcfd-9e3576cebe31, 2026-07-18); originally chosen over Pinecone (cost at white-label scale), Weaviate (heavier), pgvector (no Postgres) for a ~100–200k-vector corpus, bound to localhost as hardening (source: cc_sessions/3d06ecba-253e-b3a6-0995-642eebbef621, 2026-07-18; corroborated across 8 sessions) - Qdrant ops lessons: sync direction must follow where truth lives — the weekly VPS→local sync restored an obsolete 387-point snapshot over the local 1,657-point rebuild; semantic recall can’t surface decisions from a still-active session (ingestion at session end); nightly-backup failure root cause (failing six nights straight since 07-13): Hermes cron scripts have a hard 600s timeout, and the ~230MB nightly payload at the box’s real ~500KB/s upstream is structurally AT the ceiling — the failures started when the collections grew past the threshold; the prune was never the problem (the daily-backup cron had separately errored on an unbounded Drive prune loop vs a 265-file backlog — fixed with a lock + 40-delete/run cap); fix design: gzip snapshots before upload and skip the four static producer-less corpora (smk9_sms/voice/client_stats/pre_hermes — ~140MB of identical bytes nightly; skip when the point-count signature is unchanged, weekly forced full pass), dropping nightly payload to ~40–50MB ≈ 2 min; the cron path was made upload-only (prune 0) with prune/batch-delete moved out-of-band; prune keeps newest 7 per collection; mandatory post-ingest verification (hash/payload sample + recall test) after any —reset; same session added a Kimi Route-B usage meter to spend_watch.py — daily+MTD tokens read from state.db, dollar estimates only via KIMI_*_USD_PER_M env rates, never invented (source: cc_sessions/5e046e91-48bb-b741-0db6-0451eaf6024d, 2026-07-12; corroborated across 8 sessions; also cc_sessions/5aa78346-ac45-87b5-25a7-71ba5ff9ed10, cc_sessions/7be040c7-57c0-5885-f544-66fc299767cf, 2026-07-18)
- Ingestion pipelines: cc_ingest.py (CC transcripts from ~/.claude/projects//.jsonl), hermes_ingest.py (CC-027, Hermes state.db → hermes_sessions), claude_ai_ingest.py (full-content Claude.ai export, 400-word chunks / 50-word overlap, all-MiniLM-L6-v2, idempotent MD5 point ids, → pre_hermes_sessions); compaction cannot destroy the Second Brain — ingest reads the full transcript from state.db regardless of live-prompt compaction; recall cadence fix (2026-07-12): both Vector Brain Ingest crons (CC + Claude.ai, and Hermes) moved from daily-5am to every 20 minutes, plus a throttled
_maybe_ingest_recentbackground refresh in the Hermes hook (fires on turns, ≥8-min throttle, never blocks) — closing the gap where continuity was fresh but Qdrant recall lagged 24h across CC↔Hermes handoffs (source: cc_sessions/ea696444-06be-1cb3-cb39-ae908bd5516f, 2026-07-07; corroborated across 5 sessions; also cc_sessions/05f7bae9-fa04-e572-c233-7d6898bdcde8, 2026-07-12) - Knowledge confidence routing: 0.6–0.8 → Basic Memory wiki only; <0.6 → quarantine to alexandria/rejected/ with a reason (source: cc_sessions/756b4ca2-1ad6-5f9a-7f6f-cc5266bb351a, 2026-07-07)
- Original RAG-architecture decision with rationale (2026-07-11): use Qdrant native (qdrant-client + sentence-transformers) as the core retrieval engine — already running, ~8ms p95 self-hosted, Apache 2.0; avoid Pinecone (cost/lock-in) and Chroma (limited scale); LlamaIndex/LangChain only for higher-level orchestration/routers if needed; DSPy later for prompt optimization; collection-routing matrix (smk9_sms → client comms, smk9_voice → transcribed notes, pre_hermes_sessions → prior agent memory, smk9_client_stats → metrics) with multi-collection filter + RRF score fusion; recommended lightweight cross-encoder re-rank on top-20 (~50ms); latency budget <200ms retrieval end-to-end (all-MiniLM-L6-v2 embed <20ms, hybrid 384-dim search <50ms) (source: hermes_sessions/caf9fca6-7ac9-7686-8e68-ed01871231d7, 2026-07-11)
- Audit chunk D-1 (2026-07-18): the icm_docs collection is now in cc_recall by default — a labeled “constitution” line with a calibrated 0.25 score floor (canary 5/5); a merge-path bug that silently dropped sessionless payloads was fixed, and sync_icm.py got a non-fatal post-sync re-ingest hook so the mirror refresh never blocks (source: cc_sessions/648ec400-9251-b39a-ad30-7a541d51ab93, 2026-07-18)
- Viewer/tooling picks: Quartz v5 builds the second-brain graph site (auto-rebuild watcher, quartz-second-brain.service on :8080, /home/eco/second-brain-quartz, brain.soundmindk9.net HTTP 200); Logseq file-mode was the earlier desktop pick but is risky (DB-version split); SilverBullet noted (live server, object graph since 2.9); Markmap per-document only; Athens abandoned; notes taxonomy: encompass, concepts, personal, queries, decisions, comparisons, smk9, raw, tech (source: cc_sessions/678bbf29-c3a6-b6f6-5e56-af17b6c8420f, 2026-07-07; corroborated across 5 sessions)
- Keystone decision (Phase D, 2026-07-18): “separate-but-synced” — ICM stays the Elliott-gated constitution on Drive stable-IDs; the Second Brain becomes its retrieval layer with ICM docs finally ingested; open question logged: does D-048’s on-demand-layer intent resolve through vector recall, and do ICM + Second Brain converge or stay separate-but-synced; peer-review policy: LLM panel for scope-level plans only (source: cc_sessions/8ba41dea-79d7-1ae3-87a9-8824169c9f5e, 2026-07-18; corroborated across 3 sessions)
- Voice dataset lineage: Google Voice parse produced smk9_conversations.txt (~9MB, 9,331 conversations); pinned decision — no embeddings until Samsung SMS + WhatsApp exports are merged; single-pass data strategy (Elliott’s rule): populate DB starting with contacts, then attach conversations — never parse the large exports twice (source: pre_hermes_sessions/0235cc42-e48f-ddb2-8a33-e46c7c985d75, 2026-05-01; corroborated across 4 sessions)
Hermes runtime & config
- Hermes config gotchas:
approvals.modeonly accepts “manual”/“off” (“auto” silently falls back — though a 2026-07-02 session recorded “auto” as the intended permanent default, a config contradiction);approvals.cron_mode: denyauto-denied tool calls in agent crons causing silent failures; cronscript:values auto-prefix ~/.hermes/scripts (writing “scripts/foo.py” double-prefixes);no_agent=Truestdout IS the Telegram message — separate logging (stderr) from deliverable output; relay [CC] tasks fail under cron because PATH lacks ~/.local/bin; jobs must live in the ACTIVE scheduler instance’s jobs.json (the HERMES_HOME migration stranded Jake’s cron); systemd TimeoutStopSec must cover the drain timeout; gateway restart must run from outside the gateway process (source: cc_sessions/4626989b-a6c6-0ba4-372e-63c77629b942, 2026-07-10; corroborated across 8 sessions) - Hermes runtime gotchas: scripts need the hermes venv python (system python3 lacks qdrant_client); PEP-701 nested f-strings pass system 3.14 py_compile but fail the older venv runtime; patch remote files via base64 (avoid backtick substitution in SSH heredocs); send_path_degraded can stick after a slow deleteWebhook → gateway restart; send_message tool broken (missing slack module) —
hermes send -t telegramworks; execute_code blocked in unattended cron runs; email/IMAP polling chronically times out (fixed root cause: raw imaplib with no socket timeout → socket.setdefaulttimeout); Firecrawl plugin fails to init; shell gotcha: a launched command that begins withpkill -f <script>matches its own command line and kills its own shell — surfacing as exit code 144; relaunch without the self-kill prefix (source: hermes_sessions/8af01330-7a76-f1fc-d368-7af9b1b9c45f, 2026-07-16; corroborated across 7 sessions) - Hermes web-search architecture: all 7 search providers are plugins (brave-free, ddgs, searxng, exa, parallel, tavily, firecrawl) behind a registry dispatcher;
ddgsis free with no API key (justpip install ddgs),brave_freeactivates only whenBRAVE_SEARCH_API_KEYis set; the C2-11 fix installed ddgs and pointedweb.backendat it so Firecrawl’s “missing direct config and tool-gateway auth” init failures stop (source: cc_sessions/263c964d-5039-be3c-e159-1a5d7f90cce6, 2026-07-10) - Wayland GUI automation on eco-chi-001: the desktop is Wayland with Xwayland serving
:0;DISPLAY=:0+XAUTHORITY=/run/user/1000/.mutter-Xwaylandauth.*connects (both Hermes and desktop run as eco/uid 1000) — the cookie filename is randomized per login, so it must be globbed, never hardcoded (source: cc_sessions/136c6b37-ea83-ff19-800b-e61dac206718, 2026-07-11); the mini also has a live desktop for interactive auth:DISPLAY=:0 xdg-open <url>opens OAuth/login pages directly on its screen when copy/paste from Elliott’s terminal is not possible (source: cc_sessions/3e30ecdb-0683-600a-eb5c-1df183f6d59e, 2026-06-23) - Env-var pitfalls: eco profile uses GMAIL_APP_PASSWORD (not EMAIL_PASSWORD); delivery channels need explicit chat ids (telegram:1768792577 — Elliott’s user ID, identical for any bot in 1:1 DM; bot separation is by token); June VPS-script breakage was hardcoded /root/.hermes paths permission-denied for eco (same legacy-path failure class: Hermes agent-hooks hardcoded to /root/.hermes silently no-opped on eco-chi-001 where the gateway runs as eco — the checkpoint pipeline and the never-created resume.py both failed this way; fixed under TASK-CC-007 Part B; source: cc_sessions/53cfd955-2787-4c64-1b00-dc63a813f0d1, 2026-07-07); VPS gateway triage lesson: three hermes gateway services coexisted —
hermes-cc(the live bridge),hermes-gateway(failed/disabled), andhermes-gateway-reportscrash-looping with NRestarts=1078 while holding ports 8642/8644 — so a port-holder is not necessarily an orphan (source: cc_sessions/584b18d0-bb3e-ec4f-e786-23300039a7ee, 2026-07-08); VPS gateway runs as user smk9 reading /home/smk9/.hermes/.env (setup wizard writing the wrong file caused a multi-session key-failure loop); re-enter keys viahermes setup, never hand-edit .env; hermes.service ExecStartPre pkill frees port 8642 (source: hermes_sessions/9b3f6566-9f26-01f5-3c8e-4b159620d03d, 2026-06-25; corroborated across 5 sessions) - Hermes compaction failure traced to a removed model id: summary model grok-4-fast returned 404, forcing deterministic-fallback compaction; the auxiliary compression model (Haiku, 200K ctx) auto-lowers the session compression threshold to its own context size (source: hermes_sessions/b470cd51-65ee-1814-f741-5a506675082b, 2026-07-16)
- Fix upstream code durably at extension points; never patch the installed package (
hermes updatewipes it) (source: cc_sessions/b8aef051-76f5-72e5-e3f9-0c8870a6590a, 2026-07-08) - MCP wiring:
hermes mcp servenatively exposes Hermes as an MCP server; Claude.ai MCP connectors use POST StreamableHTTP, NOT SSE (hard-won); always add -m timeout when curling SSE; permanent URL https://smk9-hermes.duckdns.org/mcp replaced the trycloudflare tunnel; port 8765 deliberately not publicly exposed; hermes-mcp-proxy intermittently dies after VPS restarts — systemctl restart restores the connector in seconds; gateway ~10-min inactivity sleep needed a wake-ping protocol (or timeout=0) (source: pre_hermes_sessions/08667564-f8b0-496c-66ed-1845895fc8ac, 2026-05-14; corroborated across 5 sessions) - Drive tooling: google_api.py gained a real
drive update(overwrite in place by fileId — duplicates impossible by construction), propagated to stale script copies — the propagation gap was the root cause of the “broken” documented cc_queue Drive sync:drive updateexisted in the local skills copy but the installed /usr/local/lib/hermes-agent copy (Jun-1 build) and both profile copies (profiles/cc, profiles/reports) were stale and lacked it; all four copies made md5-identical, and the mandatory pre-write step (download live canonical + diff before writing) caught that Drive had silently lost the entire TASK-CC-COST-001 block to a prior stale sync; Hermes also maintains its own/home/eco/.hermes/cc_queue.mdcopy which had diverged from the repo copy — TASK-CC-COST-001 existed only there (sources: cc_sessions/21073071-7c94-e410-4df0-1488b7bf7855, 2026-07-18; cc_sessions/1b2ddce9-6ee4-2cee-3a11-bb39f1f9bef7, 2026-07-15); google_api.py is not git-tracked anywhere — it’s a Hermes-install file whose changes go live in place, captured only by the skills-curator backups (source: cc_sessions/136c6b37-ea83-ff19-800b-e61dac206718, 2026-07-11); more gotchas:drive downloadauthenticates with per-user OAuth, not the service-account ADC (credentials set up for root on the VPS did not transfer to eco on the mini), drive search requires--raw-query(a positionalname='FILE'query is treated as fulltext and returns HTTP 400) (source: cc_sessions/6ce2f87b-3317-63e5-96fc-401765fab831, 2026-06-23), and an MCP-read copy of a Drive file carries markdown escaping — for in-place edits, download the RAW file via google_api.py, apply targeted edits, thendrive updatethe canonical fileId (source: cc_sessions/616757ee-0b63-4818-577c-9eb987639b05, 2026-07-17); Drive MCPsearch_filesrejectsname=(usetitle contains), lags on brand-new files (OAuth CLI sees fresh writes); pre-Hermes lineage of the stable-fileId doctrine: file IDs change on every create → search stable parent folder by title, newest-by-modifiedTime, never >5KB inline tool params, never auto-decode >2MB; a Google Apps Script cleanup job (every 6h) auto-trashed duplicate master-doc versions (source: cc_sessions/da80b054-339e-8c77-da53-4a0266b744ad, 2026-07-18; corroborated across 6 sessions) - ICM memory discipline: WORKING tier (MEMORY.md/USER.md, ≤2-line entries) only for what must survive session gaps without Drive; each Hermes session reconstructs Memento-style (system prompt + MEMORY + USER → BRIEFING.md → Firestore session → Drive master → skills); three-tier decision lifecycle: BRIEFING §Active Decisions → SMK9_Build_Progress_Book.md (permanent, rewind demarcation) → Lessons; protected files (append-only Session_Archive.md) exempt from ALL size audits/compression; BRIEFING at ~22KB was trimmed toward 12KB (~36% base-cost saving); sync_icm.py’s 6am mirror was failing on OAuth refresh (offline fallback ~24% stale) → Phase C fixed it to read by canonical fileId, every auto-fix tagged with an exact undo path (source: hermes_sessions/26b8d329-d281-7fc7-4b0b-8fa98216afb6, 2026-06-25; corroborated across 6 sessions)
- Gate-violation telemetry: /root/.hermes/data/gate_violations.json written by session_start.py:log_gate_violation(), read+reset by the daily audit; ICM stage pipelines apply to bounded sequential tasks, NOT daemon agents (Kobi, Spider-Man) or automated routing steps (source: hermes_sessions/68cd0bb8-31a0-51a8-8447-055e68b30049, 2026-06-29 + 35622a4a-88ae-c6ee-dec6-d627e834aa2f, 2026-07-07)
- Cost tuning (June 2026): skill-creation nudge interval 15→75; kanban dispatch safety net 4h; Kanban dispatcher stays disabled until the Relay is designed and its cost model understood; token-burn tells: multiple Anthropic receipts/day, spend-threshold emails, untriggered running tasks; session guardrails: auto-compact backstop at 100 messages with an 80-message Telegram warning (CC-018 pre-compact hook verified end-to-end) (source: hermes_sessions/241f2f06-3893-a2fe-92fb-ee0239948243, 2026-06-25; corroborated across 3 sessions)
- API→CLI audit (2026-06-23): Tailscale lock not needed; Qdrant via curl localhost is correct; Firebase local Admin-SDK backup right (CLI export needs GCS); Cloud Functions https.request has no CLI equivalent; google_api.py already routes Drive to the gws CLI (source: hermes_sessions/0f39901c-1d78-0c8c-0274-4c0c3d953a24, 2026-06-23)
Security posture & audits
- 2026-07-08 security review key findings: leaked GitHub PAT = top risk (had been embedded in git remote URLs + a gitconfig insteadOf rule → “secrets never in URLs/config” codified); eco-chi-001 SPOF with no proven failover (“warm standby” was documentation-only — no working failover, no restore test); SSH password auth enabled on a box with a 0.0.0.0 API + unsandboxed terminal (RCE path); WhatsApp creds.json world-readable; passphrase-less automation SSH keys incl. one unlabeled key in root’s authorized_keys and one reaching both root+smk9; old ADMIN_KEY in git history; Ecosystem_Config.md carries a plaintext webhook secret; Drive sync OAuth token holds full drive scope (source: hermes_sessions/e9782e9b-0d8c-8a9a-bfd4-30cef92541b4, 2026-07-08; corroborated across 6 sessions)
- Confirmed security-audit findings (2026-07-08 Hermes-side review, all CONFIRM): unauthenticated PII read+write on real client intake drafts (top real risk — breach + tampering); outbound-email gatekeeper completely absent at the time (regulatory + attorney exposure); QuickBooks MCP invoice/payment creation reachable by the LLM with no approval (direct financial exposure); HMAC secret duplicated in a Drive-synced file and bound to 0.0.0.0; Bitwarden disabled with plaintext .env as the actual secret store; OAuth refresh broken with 6kB config drift; untrusted file contents injected under —dangerously-skip-permissions; systemic misses: no input sanitization/schema validation on intake formData, no rate-limit or phone-proof on /draft or /save-step, no git-secrets/pre-commit scan, no Firestore TTL purge (later built), webhook listener on 0.0.0.0 without mTLS or IP allowlist (source: hermes_sessions/2cd84181-c077-f6c6-b581-cd49c030952d, 2026-07-08)
- Credential-hygiene flag still open: the old Windows laptop’s D: drive holds stale Windows-path references plus a loose Firebase key — low priority, but a live credential sitting outside the managed machines (source: cc_sessions/421e87b1-8546-e8ec-e50c-091243ea7a58, 2026-07-07)
- Hardening applied (C2 series): SSH password auth disabled both hosts (VPS root key-only) — then temporarily re-enabled 2026-07-09 after UFW tightening locked Elliott out, to be re-disabled once key auth is confirmed on both laptops; public 22/tcp ufw rules deleted (Tailscale/LAN only; the open port explained observed brute-force traffic); dormant root SSH key removed with reversible backup; local secret files (.hermes/.env, google_token.json, Firebase admin key) 644→600 (hermes-gateway runs as eco so reads survive); flagged: webhook HMAC secret plaintext in config.yaml with listener on 0.0.0.0:8644 (source: cc_sessions/5c43c006-6e4d-df6f-15d1-cd0c4d02b1e7, 2026-07-09; corroborated across 6 sessions)
- Credential-expiry policy: server-side keys in 600-perm files get “never expires” — the ecosystem has a documented history of expiring credentials silently breaking things (FIREBASE_TOKEN); instant revocability covers compromise risk (source: cc_sessions/34962f77-4bf9-b4d5-7a67-50ff17eeb717, 2026-07-12)
- Multi-LLM adversarial audit as working QA: MoA = Grok + Kimi + Sonnet with 2-of-3 different-family critical-finding vote (LLM aggregator replaced by deterministic code tally); found what single agents missed — payroll-mutating endpoints with no approval gate, Fireman writing to prod Firestore every run, an agent report falsely claiming Gatekeeper enforced; reviewers must be independent, non-leadingly prompted, instructed to refute; reviewer family ≠ author family; mandatory Phase-0 fresh live snapshot; findings without a reproducible command auto-dropped (source: cc_sessions/07518513-d868-302c-20dc-3e62d1199caf, 2026-07-08; corroborated across 6 sessions)
- Audit blueprint hardening (2026-07-18): nervous-system circuit-breaker + self-test, explicitly defined verifier (veto + read-only state access), essential-writers list, SLOs, token budget ceiling; design principle from the CC-025 critique: “validation theater exceeded validation substance” — prefer verification primitives over process ceremony (source: cc_sessions/2932a0e2-88ab-ec6e-0a32-77012a376651, 2026-07-18 + b40f9b6c-48b1-1b53-c0e8-5e6d72c8b962, 2026-07-10)
- Phase D audit toolchain (green, commit 95e746d): PII scanner with 4 detector classes + a 32-name/39-phone roster pulled read-only from Firestore (fail-closed, Elliott allowlisted); exclusive writer with scanner + batch_id + protected collections; PII regime ruling: sample-based “PII-safe” claims rejected (a ~13-record sample over 28,697 points proves nothing; self-declaring violates no-self-grading); D-3 write-path: synthesis is the ONLY writer with the scanner welded inside, alexandria cron suspended per-chunk with can’t-forget-to-restore semantics, daily committer as tripwire; soak/fault-injection tests hard-sandboxed from outbound sends (LESSON-007 class) (source: cc_sessions/b453e6a1-964d-31bd-81c5-f613df93a3cf, 2026-07-18; corroborated across 4 sessions)
- Relay input-injection guard (M5): trusted-path allowlist + prompt-injection marker scan over inlined task inputs — verified rejecting /tmp-sourced inputs and “Ignore all previous instructions” payloads; H4 Gatekeeper hook: QuickBooks financial mutations DENY by default, reads ALLOW, QBO_WRITE_APPROVED=1 override; Dr. Manhattan isolation: separate OS user
drmwith root-owned read-only bind mounts (plain permissions failed — Hermes self-chmods); hermes_critical encrypted backup lane (tar.gz.gpg ~70M) + dm_heartbeat cron */15 (source: cc_sessions/1c2a3f42-ff9b-2f08-201b-25cdf5b66d0b, 2026-07-08; corroborated across 4 sessions) - Google API key scoping: Maps key keeps Places (New), Routes, Geocoding, Address Validation (+optional Air Quality); browser-facing Maps JavaScript API belongs on a separate key; Places uses a dedicated restricted key (public data), not an OAuth client ID — separate keys keep blast radii independent; address lookup (CC-031) closed by enabling “Places API (New)” on project
sound-mind-k9-business-buildwith a key stored asGOOGLE_MAPS_API_KEYin~/.hermes/data/smk9_keys.env— gotcha: enabling the API ≠ having a key; verified live viagoogle_api.py places search, a one-command, primary-source, CAPTCHA-free path (sources: cc_sessions/3c88bda2-6961-096f-1a32-383d53a7ae4a, cc_sessions/565db398-208f-5048-4ae6-4226fa59e82a, 2026-07-11) (source: cc_sessions/fc797de8-6fba-4fb3-d3b4-94ec6b6d5d0d, 2026-07-11; corroborated across 3 sessions) - DR posture: SMK9_DR_Runbook.md in the Drive anchor folder (architecture, backups, passphrase location, rebuild steps, RTO/RPO); VPS dead-man’s-switch cron (*/5) Telegrams Elliott directly via Bot API if the mini goes dark ~15 min, announces recovery; daily gpg AES-256 ~/.hermes backup to Drive (passphrase in a 600 file mirrored to VPS; Bitwarden third copy judged unnecessary); VPS standby lacks ollama, the Relay (host-locked xAI OAuth), voicebox; RPO ≤1 day / RTO ~minutes for the client path (source: cc_sessions/df7ca86a-a124-1f82-7a94-1861ab074ca7, 2026-07-09; corroborated across 5 sessions)
- OAuth URLs captured from tmux panes get truncated by line wrap — join wrapped lines (tmux capture -J) before opening the browser; PuTTY does not implement OSC 52 (clipboard-over-SSH impossible; write files instead — supersedes the earlier 07-11 attempt that added tmux
set-clipboard on/OSC 52 to ~/.tmux.conf expecting PuTTY copy-out; source: cc_sessions/565db398-208f-5048-4ae6-4226fa59e82a, 2026-07-11); Google Workspace tooling on the mini uses user OAuth, not the service-account ADC (source: cc_sessions/61e3985d-b5d2-c3b7-b8af-bb1ae01dea97, 2026-07-18; corroborated across 3 sessions)
Weather/alerting stack
- Rain detection cascade (approved): Tomorrow.io radar intensity PRIMARY; NWS KMDW gauge (~5 mi) last-hour precip SECONDARY (independent-failure); Open-Meteo model TERTIARY; double-validation gate holds “rain stopped” while the gauge still shows precip; radar-primary morning forecast with nowcast override (source: hermes_sessions/289004a6-aa02-e3c9-0587-5ccd71ec1d77, 2026-07-10; corroborated across 4 sessions)
- rain_watch failure chain → CC-030 final rules: state/log paths were hardcoded to the dead Seagate (silent fallback to dumb rain_monitor.py since Jul 2); RAIN_PROB_THRESHOLD=50 was ignored whenever a WMO rain code was present (1am/34% false alert);
log()on stdout spammed Telegram (no_agent stdout IS the channel → stderr); deeper false-alert root cause: the live/home/eco/rain_watch.pywas never version-controlled — earlier “fixes” only committed queue files while the real code sat untracked; now synced into the repo (scripts/rain_watch.py, replacing a stale 610-line copy with the real 1,362-line script), with non-radar (NWS/Open-Meteo) forecasts never sending Telegram (radar confirmation required), a zombie-event sweep before alert logic, and the NWS probability floor raised to 60% (source: cc_sessions/904d8086-eac8-85e3-be3e-119f7e892f41, 2026-07-11); final: radar confirmation required for EVERY alert, NWS threshold 60%, WMO filters need ≥30% floor, zombie events deleted on expiry/startup, requires TOMORROW_API_KEY, briefs quiet below thresholds; 11/11 unit assertions; runs on eco-chi-001 with the VPS copy cold-standby (source: hermes_sessions/e0984dc0-6977-b8da-f526-bef0de9eade8, 2026-07-09; corroborated across 6 sessions) - Alerting policy: routine maintenance crons silenced — only non-zero exits alert; email watchdog subject-matching tightened (“domain expir”/“domain renew”) + 7-day seen-state dedup (~/.hermes/data/email_watchdog_seen.json); two-stage watchdog pattern: local-LLM scanner cron feeding a Sonnet reasoner via
context_from; true zero-cost scan = no_agent script (source: hermes_sessions/978678a4-3886-3715-bc83-4c2e770a480f, 2026-07-09; corroborated across 3 sessions)
Home hardware & streaming lane
- Wyze camera constraints: the bridge must run on the local LAN — VPS auth succeeds but streams fail (192.168.1.x unreachable from internet); Wyze password contains
!,,,/— bash double-quote history expansion breaks it, use single-quoted heredoc env files or the bridge UI at localhost:5000; myQ cloud API dead since Dec 2023 (pymyq broken, HA integration removed) — ratgdo hardware (~$35) + local MQTT is the only reliable garage-door path (source: hermes_sessions/48e04182-32ef-4fc0-947a-b23f5a43d2f9, 2026-07-01; corroborated across 4 sessions) - PIA/network fixes: NetworkManager-based fixers cannot clear a stuck PIA kill-switch; old fix-network.sh had a dead dhclient path (not installed on Ubuntu 26.04); durable fix = NM dispatcher hook /etc/NetworkManager/dispatcher.d/90-pia-roam (must be root:root 0755 — NM refuses group/other-writable hooks) + switch to WireGuard (native roaming); don’t churn PIA regions rapidly — it wedges the daemon and drops all internet (source: cc_sessions/2370a739-5803-63a5-0800-75f7008f02a9, 2026-07-17; corroborated across 4 sessions)
- Streaming stack: every torrent byte packet-verified inside the PIA WireGuard tunnel (kill switch drops streaming if PIA drops); leak protection is two independent layers — PIA’s
vpnonlycgroup (classid 1384, joined by the/usr/local/bin/qbittorrent-vpnonlywrapper: protocol-agnostic, survives VPN protocol changes) plus qBittorrent’s ownnetwork_interfacebinding, which is interface-specific (tun0 → wgpia0 on the WireGuard switch) and fails safe — dead, not leaking — when stale (source: cc_sessions/3f443aea-b094-d2c7-50a8-27d6b314c390, 2026-07-17); Stremio web at 100.85.58.103:11471, streaming server 11470; Stremio desktop (flatpak com.stremio.Stremio, with codecs runtime) is installed on the mini and launches on the TV’s Wayland session — cross-device sync between desktop and web deliberately tabled by Elliott (sources: cc_sessions/8f6dedca-08ce-801d-2ccf-1f3c58ac6a55, cc_sessions/92f2f462-1e42-4e79-dd99-f9470fe8311d, 2026-07-17); P2P slowness root cause after the path repoint: the “turtle mode” alternative speed limit (alt_dl_limit= 10240 B/s ≈ 0.1 Mbps) was capping every torrent; qBittorrent 5.x renamed theresumeAPI endpoint tostart; the wgpia0 tunnel runs MTU 1200 (source: cc_sessions/6485e4a1-1954-7c73-8fb5-c4dfc5946c16, 2026-07-18); qBittorrent tuned 12 parallel / 800 connections / no caps; debrid landscape — French/Arcom crackdown gutted RD/AllDebrid/Debrid-Link (RD cache purge hurts new releases; “a waste of my $16”), TorBox caches better but US checkout broken, Premiumize recommended for US signup; raw P2P is inherently slow (Stremio downloads sequentially); Elliott bought RD 180 days (2026-07-17, business card) and wired it into Torrentio/ffmpeg — after earlier deciding to let it lapse; selection criteria: outside French jurisdiction + working US card + Stremio integration (source: cc_sessions/f1fc4631-bdeb-b0b2-d0b9-804811c8c288, 2026-07-17; corroborated across 8 sessions) - Media-stack provenance (Indy finding, resolving Elliott’s “I’d never have chosen that username” suspicion): the *arr/Jellyfin stack was built by earlier CC sessions — jellyseerr container created 2026-06-25, jellyfin/radarr and the rest 2026-07-05; operational gotcha from the same dig: Jellyfin’s admin API refuses to reset your own account’s password (HTTP 400) — DB-level reset is the reliable path (source: cc_sessions/5e238583-8d73-fa8c-6f49-963c024a3b45, 2026-07-18)
Pre-Hermes foundation decisions (Apr–Jun 2026)
- Firebase project sound-mind-k9-business-build, Firestore nam5 multi-region (chosen over nam7 for auto multi-DC replication); full schema (clients, dogs, vaccinations, bookings, availability, stay logs, payments, config) designed 2026-04-25; Layer 0/1 complete 2026-05-07 (Admin SDK auth, Blaze — Spark can’t make outbound HTTP from Cloud Functions, 14 composite indexes); build sequenced as a 9-layer dependency stack (Firebase → forms → dashboard → historical data + voice profile → vector DB + Hermes → WhatsApp → payments → revenue intelligence), build in order (source: pre_hermes_sessions/6a6036e0-1d31-6468-3043-3b5e27f82017, 2026-05-02; corroborated across 6 sessions)
- Firebase web API keys are public by design — protection comes from security rules, not key secrecy; public write rules added for intakeForms and emailQueue (source: pre_hermes_sessions/0f9983a4-c665-4a27-f4c3-27ad6365eb3f, 2026-05-07)
- Messaging-stack lineage: WhatsApp Business API with 360dialog as BSP (~$10/mo, passes Meta rates); ManyChat as no-code flow hub, Wati passed over — then ManyChat/Wati rejected for white-label after Elliott caught a contradictory recommendation; own-the-stack doctrine: WhatsApp is only the channel, Claude the brain, middleware swappable (keeps white-label resale possible); Baileys (unofficial) vs 360dialog conflict flagged as separate lanes; interim: manual WhatsApp free while the automation layer is built; Meta’s Jan 2026 ban on open general-purpose chatbots suits a booking-scoped bot; target pipeline WhatsApp → 360dialog → Firebase → Hermes → Claude → response (source: pre_hermes_sessions/a4b5bea2-ccf2-4c9f-f289-8f7778a21af3, 2026-05-01; corroborated across 8 sessions)
- WhatsApp Cloud API concrete config (June 2026): Meta App “Sound Mind K9” (App ID 1548628566608467, WABA 1503485084861441); permanent system-user token required; do NOT use Marketing Messages API, mutual TLS, or the Tech Provider/App Review path; webhook soundmindk9.net/api/whatsapp/webhook (source: hermes_sessions/a660a262-aa5c-32e8-fee7-5939f873eb09, 2026-06-25)
- E-signature: signature_pad JS + Firebase Storage + timestamped audit trail + PDF — legally binding in Illinois (ESIGN + 5 ILCS 175), replacing DocuSign at $0/mo; an email-sniffer Cloud Function auto-captures city license numbers from forwarded confirmations (source: pre_hermes_sessions/66431067-3ac0-b38a-bf7a-0cf484e356f9, 2026-05-07)
- Tool-routing doctrine (SOUL.md): Hermes = persistent memory, WhatsApp automation, scheduled execution, real-time client interaction; Claude.ai = complex reasoning/planning; Claude Code = heavy builds; escalation rule: if a task needs a CC build step, Hermes messages Elliott and stops (violated once — intake HTML built inside Hermes, June 2026 billing spike) (source: pre_hermes_sessions/efe83946-1d86-b947-76a4-52ab6411bea1, 2026-05-14; corroborated across 4 sessions)
- CLAUDE.md at repo root adopted (2026-05-06) as CC’s native session memory bridging claude.ai ↔ Drive ↔ Claude Code; Claude↔Hermes task-passing designs considered: claude_inbox.md polled by cron vs authenticated /claude webhook; Qdrant proposed as shared vector memory between Claude and Hermes over slow Drive-mediated sync; volatile state isolated into a small state file instead of bloating master docs (source: pre_hermes_sessions/95e76225-eb2a-8232-a074-5b2d4e0afa0e, 2026-05-06; corroborated across 3 sessions)
- Early tech stack (pre-Hermes): Node.js + Bun v1.3.13; Docker planned for Firebase emulator, Node, PostgreSQL, Qdrant; critical path VPS up → Hermes install → Firebase Admin SDK; dev env was C:\Users\darka\smk9-app (Node v24.15.0, Firebase CLI 15.15.0) before the mini; ElevenLabs voice cloning planned for the AI concierge + training-video narration, Captions.ai for mobile short-form (source: pre_hermes_sessions/ffcf8030-3015-bc6a-0407-e57a85b07648, 2026-05-06; corroborated across 3 sessions)
- Voice interface build order (research): Phase 1 Telegram voice → Hermes → Basic Memory → TTS reply (zero new infra); Phase 2 FastAPI PTT service + Caddy HTTPS (mobile getUserMedia requires HTTPS; minimal path = MediaRecorder→opus→POST + PWA manifest); Phase 3 openWakeWord on the mini’s mic (source: hermes_sessions/ed2751c0-31a6-8ebe-aa98-3e80508d5272, 2026-07-07; corroborated across 2 sessions)
- Skip-decisions on record: Gemini in Firebase skipped (only unlocks AI Logic SDK, nothing for Firestore); Route Optimization API unnecessary (Routes API optimizeWaypointOrder covers it); Maps Grounding Lite too pricey (source: cc_sessions/9de64c29-f170-f8d7-cbae-5c578e3d3cc3, 2026-06-23)