Skip to content

Vault enrichment: idempotent export, note titles, named cluster hubs

Goal: make the Obsidian vault genuinely usable as a daily surface: graph labels become real titles, clusters become named + readable hub pages, and the export stops generating churn that fights the sync bridge.

Status: SPEC 2026-06-12 — for Aaron's review. PR A (idempotent export) pre-approved to ship same night; B and C build after review.

Context (why now): the 2026-06-12 sync incident — the exporter rewrites every file every pass, so each export looks like a 2.8k-file burst to livesync-bridge; a burst crashed the bridge mid-push, and its crash-replay pulled incomplete revisions back OVER the fresh vault (4,934 reverts in 100 min, root-owned files, conflict storms on both devices). Bursts are the trigger; A removes them. B and C then multiply file richness without multiplying churn.

Three PRs, strictly ordered:

PR A — idempotent export (compare-before-write) [SHIP FIRST]

  • Export reads each target file before writing; byte-identical content is skipped (no write, no mtime bump, nothing for any watcher to see).
  • ExportResult gains Unchanged int; Written becomes "actually wrote". POST /export reports both — a nightly run over a quiet corpus should show files: ~0..20, unchanged: ~2800, and THAT is the verification the burst problem is gone (visible in the run ledger, per the house rule).
  • A read error falls through to writing (never fail an export because a stale file was unreadable) — but a PERMISSION error surfaces loudly, as today (the incident's root-owned files must not be silently skipped).
  • Tests: second pass Written=0/Unchanged=N with mtimes byte-equal before and after (the watcher-visibility proxy); content change writes exactly the changed file; existing determinism tests unchanged.

PR B — note titles (fixes every label in the system) [IMPLEMENTED 2026-06-12]

Schema + distill: - Migration: distilled_notes.title TEXT NOT NULL DEFAULT ''. - The distill response schema (NotesResponseFormat) gains title: 5-10 words, plain, no emojis. The write gate enforces: non-empty, <= 80 chars, rejected note on violation (same loud-failure path as other gate rules). - Titles are written ONCE at distill time and never regenerated (note content is effectively immutable post-distill; correction = tombstone + new note, which gets its own title). No drift, no churn.

Backfill (one-time, ~2.8k untitled live notes): - POST /titles/backfill?limit=N — batches untitled live notes through the existing chat tier (local qwen; falls back per LiteLLM routing), same bounded-batch + per-batch-commit pattern as EmbedNotes/dedup backfill. Response reports titled/failed counts; convergent (titled notes skip), so re-running after any failure is safe. - Title-only generation prompt sees the note content + context; output is schema-constrained (single string field) — no parsing heroics.

Export: - Filename slug comes from title when present, else content (today's behavior): note-1234-tdarr-worker-cap-during-backups.md. - One-time rename churn across the corpus when the backfill lands: with PR A in place this is ONE controlled burst (rename = remove + write), done once, ideally while devices are idle. Coordinate like the initial sync. - ExportResult.Titled count surfaces coverage in the run ledger.

PR C — named clusters + hub pages (the synthesis pass, first slice) [IMPLEMENTED 2026-06-12]

Naming (LLM judgment on top of the deterministic structure — the kNN/ union-find layer from the links spec stays untouched): - New table cluster_names(member_min_id BIGINT, membership_hash TEXT, name TEXT, summary TEXT, model TEXT, created_at, PRIMARY KEY (member_min_id)) — the cache. membership_hash = sha256 of the sorted member note ids. - POST /clusters/name (a scheduled task, runs BEFORE the nightly export): recomputes clusters (same code path as export), and for each component of size >= 3 whose membership_hash has no cached row (new cluster or membership changed), one LLM call over the member notes -> short kebab-case name (<= 5 words) + one-paragraph digest. Size-2 components keep the anonymous id (naming 150 pairs is noise). Response reports named/cached/skipped — auditable, and the cache table records every name with its model and inputs hash. - Name collisions disambiguate with the min-id suffix (tdarr-ops-1007) — deterministic, no LLM retry loop.

Export: - Members' frontmatter carries the NAME: cluster: tdarr-ops (falls back to cluster-<minid> for unnamed/size-2 components — absence of a pretty name is visible, never invented). - One hub page per named cluster: cluster-tdarr-ops.md — frontmatter (cluster key, member count, generated-at), the LLM digest paragraph, then a member list of wikilinks (actual filenames). Hub pages join the managed namespace (^cluster-[a-z0-9-]+\.md$ added to the cleanup regexp) so stale hubs leave the vault like stale notes do. Collision guard: the managed pattern must never match a user-created file the exporter didn't write — same contract as note-*.md today. - Each member's body gains Cluster: [[cluster-tdarr-ops]] (a BODY link, not frontmatter — frontmatter links don't reliably create graph edges). Graph effect: every cluster gets a named, clickable anchor node; opening it reads the digest. This subsumes the "tag hubs" alternative (rejected: tag nodes aren't readable/clickable into content). - ExportResult.Hubs count; largest_cluster already guards blob regressions.

Explicitly out of scope

  • Typed note-to-note links (links column) — still the full synthesis pass, still parked.
  • Per-note tags as Obsidian tags (the noisy-tag-hub problem); keywords/tags stay plain frontmatter data.
  • Graph group/color presets — per-device Obsidian config, can't ship it.
  • Obsidian -> Argus capture (separate roadmap item, design-first).

Operational sequencing

  1. PR A merges + deploys -> verify a no-op export reports unchanged~=all.
  2. PR B merges -> run titles backfill -> verify coverage -> ONE coordinated export burst (devices idle) -> graph labels become titles.
  3. PR C merges -> /clusters/name runs -> export -> hub pages appear; add Ofelia schedule name-clusters before the nightly export.
  4. Bridge stays as-is; if bursts are gone and crashes persist anyway, THEN invest in the bridge (throttle or one-way replacement) — evidence first.