Skip to content

Argus cutover: pve-claude-code (dev) -> pve-argus (LXC 107, permanent)

Date: 2026-06-11. Status: DEFERRED (2026-06-12 — until tasks module + minimal UI ship; see ROADMAP "Deferred"). pve-argus LXC is provisioned (Komodo server) but the live stack still runs on the argus-build dev box (LXC 136). Spec: docs/specs/2026-06-09-argus-design.md §8. (Condensed from the planning agent's investigation, verified 2026-06-10.)

Verified state

Dev stack healthy on .96 (compose, DB 15MB: raw_items ~3800, distilled_notes, episodes, segment_checks, distill_state 28, projects 17, gate_rejections, injections). Target pve-argus: Komodo server Ok, Periphery v2.2.0, .34, 100G /data. Both Go services serve /healthz with baked HEALTHCHECKs.

Capture-hook outage semantics (UPDATED for v0.2.0 / PR #8): LOSSLESS in the common case. Offsets now checkpoint per handled line, so an outage freezes the offset at the last posted turn and the next Stop/SessionEnd resumes from exactly there — at most one turn re-sends, and the server's dedup_key makes even that a visible no-op (duplicate=true, no new row). Remaining caveat: a session whose final event fires entirely inside the outage and never resumes loses its last increment — cut over at a quiet moment.

Decisions

  1. Stack declared in homelab-apps/komodo/resource-sync.toml (repo field points at runonyourown/argus) — one ResourceSync to rule them, spec's "fold into homelab-apps later" is now. Entry mirrors metamcp's: server=pve-argus, branch=main, run_directory="." (FIRST repo-root stack — verify Komodo accepts "."), file_paths=["compose.yaml"], env_file_path=.env, auto_pull=false, run_build=true, pre_deploy sops -d secrets.enc.env > .env.
  2. Images: run_build=true for v1 (metamcp precedent; compose already uses build: contexts). v2: CI build -> Forgejo Packages -> auto_pull when the task-runner gets its custom image (M3/M4).
  3. Data: full pg_dump -Fc / pg_restore — DB is 15MB; selective restore fights GENERATED ALWAYS identity + the immutability trigger; full dump preserves audit history (segment_checks, gate_rejections) and carries schema_migrations so the migration runner no-ops.
  4. ARGUS_PG_PORT=5432 published on .34 (LAN/Tailscale only, never routed).
  5. Mac uses Tailscale MagicDNS (http://pve-argus:8081); requires tailscaled in LXC 107 (UNVERIFIED — pre-flight before Mac repoint). LAN clients use http://192.168.1.34:8081.

Phases

  1. Pre-flight: pve-argus reachable + /data + docker; sops + age key at /etc/komodo/sops-age.key (or inside periphery container); resolves git.aaronbrazier.com; reaches LiteLLM .28:4000 + proxy.golang.org; baseline row counts on dev.
  2. Secrets (BLOCKING GAP: secrets.enc.env is stale — 5 of 10 keys, no ARGUS_LITELLM_KEY/ARGUS_CAPTURE_TOKEN/EMBED/DISTILL/PG_PORT/ROUTINE_PREFIXES): rebuild with all 10 keys ($EDITOR, never echo), NEW random POSTGRES_PASSWORD (dev's is throwaway), SAME ARGUS_CAPTURE_TOKEN (clients keep working), ARGUS_ROUTINE_PREFIXES="Run the Tdarr HEVC monitor" (added 2026-06-11, PR #13 — not secret but must survive the sops round-trip); sops -e -i secrets.enc.env; verify grep -c 'ENC\[' = 10; decrypt-proof on 107 with count-only output. Also rm secrets.env (stale plaintext).
  3. Komodo declaration: PR to homelab-apps with the entry above; sync.
  4. First deploy against EMPTY db: debug build/SOPS/network/Ofelia before the window. Verify /healthz both services from .96, capture+recall round-trip with the real token, ofelia logs show heartbeat+distill firing.
  5. Cutover window (~15 min, ordered): stop ofelia+task-runner then memory+hub on dev -> final pg_dump -Fc -> scp -> pct push 107 -> stop app services on 107 (postgres up) -> DROP/CREATE argus -> pg_restore --no-owner --exit-on-error -> verify counts vs baseline -> start services (migrations no-op) -> repoint ~/.argus/env on .96 to http://192.168.1.34:8081 -> let a Stop fire, confirm raw_items count EXCEEDS baseline (frozen-offset turns re-sent) -> recall round-trip.
  6. Clients: .96 done in 4 (and already on v0.2.0-built binaries as of 2026-06-11). Mac: v0.2.0 IS released — download darwin-arm64 hooks + checksums from the release page, ~/.argus/env with MagicDNS URL + same token, same wrappers + settings.json blocks; verify Mac-hostname rows.
  7. Gatus: new homelab-infra/gatus/conf.d/argus.yml — argus-memory (.34:8081/healthz) + argus-hub (.34:8080/healthz), type telegram, 3/2 thresholds, send-on-resolved.

Rollback

Old stack left stopped-not-removed until 7-day soak. Revert = repoint ~/.argus/env to 127.0.0.1:8081, docker compose start old, stop new stack (no dual distill pipelines). raw captured on new meanwhile is stranded but append-only — recoverable by captured_at window. Triggers: healthz flapping, distill errors vs LiteLLM, restore count mismatch, capture failing post-repoint.

Decommission (after 7-day soak: gatus green, notes growing, both clients capturing)

docker compose down on dev (.smoke/data preserved), keep final dump 30 days. Dev box remains a CLIENT + dev checkout (harness smoke uses throwaway stacks).

Gaps found

  1. secrets.enc.env stale (BLOCKING — fixes in Phase 1)
  2. plaintext secrets.env in repo dir (delete)
  3. unmerged branch at plan time — RESOLVED: all of audit cycle 1 merged (#6-#13)
  4. no image-publish CI (accepted v1; M3/M4)
  5. raw_items dedupe — RESOLVED by PR #8 (generated dedup_key + app-level idempotency check; unique constraint impossible: 68 pre-existing duplicate groups in append-only history)
  6. task-runner/ofelia no healthchecks (accepted; heartbeat covers)
  7. run_directory="." unprecedented in the sync file (verify on deploy)
  8. tailscaled in LXC 107 unverified (needed for Mac only)
  9. argus-*:dev image tags in prod (cosmetic; fixed by v2 registry move)
  10. v0.1.0 predates current main — RESOLVED: v0.2.0 tagged + released 2026-06-11