Skip to content

Episode Segmentation for Continuous Conversation Streams — Research

Date: 2026-06-10. Method: deep-research workflow (102 agents, multi-source sweep, 3-vote adversarial verification per claim). Question: how to segment a conversation stream with NO natural session end (our case: one Claude Code session resumed indefinitely on pve-claude-code) into units for memory distillation.

Why this was needed: session_id is constant forever on this box, so "distill per session" degenerates into one infinite group. memory-os had no answer — its mechanism was per-response extraction + synthesis on an explicit on_session_end event, which never fires for a never-ending session (that is where its junk message-fragment "facts" came from).

Verified findings (vote = adversarial 3-skeptic check)

  1. [HIGH 3-0] The unit is the EPISODE, not the session. Zep/Graphiti's atomic ingestion unit is the episode, bounded so several fit in an LLM context window; it separates event time from ingestion time (bi-temporal). Everyone segments; nobody distills whole sessions.

  2. [HIGH 3-0] Variable-length episodes via a multi-signal boundary detector, not fixed windows alone. Dominant signal: topic/semantic shift. Corroborating: intent transitions, time gaps (>30 min), structural transition phrases, content-relevance (<30%), with a "when in doubt, split" bias (Nemori, Membox Topic Loom, EM-LLM Bayesian surprise).

  3. [HIGH 3-0] The infinite-stream case requires a DISJUNCTION: close = topic-shift OR hard size cap. Nemori formalizes it: T = (boundary detected ∧ confidence > σ) ∨ (|buffer| ≥ β_max), β_max = 25 messages (soft 10–15). The size cap is the guarantee that a segment always terminates even when no shift ever fires — not a fallback, the essential mechanism for a stream with no boundaries.

  4. [HIGH 3-0] Incremental, bounded distillation. Extract against a stored summary + recency window (Mem0) or per closed episode; build higher-level synthesis from already-distilled units (Generative Agents reflection trees, triggered by cumulative importance, not a clock). Never re-batch the growing corpus.

  5. [HIGH 3-0] Dedup is per-fact LLM merge: retrieve top-k semantically similar existing facts, LLM chooses ADD / UPDATE / DELETE / NOOP (Mem0; Zep entity resolution adds embedding + FTS candidate gathering). Better than blanket tombstone-and-rewrite.

  6. [HIGH 3-0] Over-extraction control, two complementary tools: (a) importance/poignancy scoring (Generative Agents 1–10, retrieval weight + reflection trigger) — we have this (the importance bar + column); (b) prediction-error filtering (Nemori, strongest result): synthesize an anticipatory schema of what the episode probably contains given existing knowledge, extract ONLY the residual it failed to predict — "predictable implies redundant." Future upgrade for our distiller.

  7. [MEDIUM 3-0, fewer systems] Derived artifacts must never re-enter raw. Keep the immutable raw log authoritative; summaries/compaction output are a separate, provenance-linked tier, never re-ingested as turns (TierMem). This names the guard against the compaction-summary self-poisoning failure mode.

  8. [MEDIUM 2-1] Continuity across cuts: post-segmentation merge — embed each new episode, search top-k existing episodes, LLM merges only same-event episodes and refuses merges across >1h gaps. Repairs episodes severed by the size cap without reprocessing the corpus.

Caveats

Evidence is mostly 2025–26 research-grade systems (Nemori, EM-LLM, A-MEM, Membox, TierMem) plus Generative Agents (2023); Mem0 and Zep are the closest to production. Letta/MemGPT, cognee, and LangMem did NOT survive verification in this pass — do not cite the MemGPT "flush" pattern from this research. Mechanisms are well-corroborated; specific thresholds (25-turn cap, 30-min gap) are starting points to calibrate on our own data.

Our design (decided 2026-06-10)

Tiered boundary detector — deterministic guarantees first, ML signal second, LLM only if calibration demands it:

  1. Deterministic (always on, no model): hard size cap (~20 turns) + time gap (>30 min) + idle-close for a session's trailing turns once the burst is clearly over. Works with zero LLM/embedding dependencies.
  2. Embedding cosine-drop (traditional ML, local, auditable): embed turns with the resident nomic model (embed tier, 768-dim); boundary before a USER turn when its embedding's cosine similarity to the rolling mean of the recent window drops below threshold. Modern TextTiling / changepoint detection on the similarity series. Every boundary stores its score (boundary_score) and reason (boundary_reason) — "why did it cut here?" is a SELECT. User-turn anchoring: topic changes are user-initiated; assistant turns follow their user turn's topic, so boundaries only open before user turns.
  3. LLM escalation (deferred): only for an ambiguous score band, only if the embedding signal proves insufficient on real data.

Distillation then operates per CLOSED episode (the open tail of a live session is never distilled); kind=turn routes through episodes, all other kinds stay per-item. Compaction-summary turns are filtered at capture (finding 7).

Related capture hardening (from the same analysis): raw-level idempotency to prevent duplicate re-capture if the hook's byte-offset state is lost (the offset-file failure re-POSTs the whole transcript; raw_items has no dedup today).