Context-Injection Ranking — what actually helps (2025-2026)

Date: 2026-06-24 Trigger: ARGUS-76 cross-encoder re-rank shipped behind a flag, then the on/off eval (ARGUS-87) measured a wash: ~+6% precision@12 but ~-10% recall@12 with bge-reranker-base. This doc is the research into why, and what to do instead.

TL;DR — the reframe

The problem is set selection, not ordering. Modern LLMs (Claude included) are increasingly order-insensitive — they reason over multi-chunk context robustly regardless of rank order. A cross-encoder optimizes order / precision@K, which is the thing the consuming model cares least about. So reranking is the wrong lever for injection-into-an-agent. The field has moved to adaptive selection of a small, high-precision set + lost-in-the-middle mitigation.

(a) Why our reranker washed — expected, not a config bug

• Domain mismatch + model size. Off-the-shelf cross-encoders degrade out-of-distribution; OOD generalization needs larger models. bge-reranker-base is the small end, and our corpus (CC sessions, dev decisions, syntheses) is OOD vs its web/QA training. Reranking a noisy candidate pool produces exactly our symptom — recall decline + high-scoring false positives. (Déjean et al. 2024, arXiv:2403.10407.)
• Rerankers optimize the wrong objective. They optimize ranking metrics (NDCG/MRR), not downstream decision correctness (CalibRAG). A listwise LLM reranker matched but did not beat a simpler approach on a real downstream metric, at +40% latency / +15% cost (Fin.ai A/B report).
• Order-insensitivity. GPT-4o / Claude 3 / Gemini 1.5 handle multi-chunk reasoning regardless of order, undercutting reranking's main benefit.

(b) Lost-in-the-middle — this challenges maxNotes=12

Injecting more notes (high recall) actively hurts generation: relevant info in the middle of a long context is under-attended — a U-shaped accuracy curve (Liu et al. 2023). A benchmark showed RAG accuracy drop 33% when the relevant chunk sat in the middle of 10. Current guidance: fewer, higher-precision chunks (3-5), don't dump 12-20.

This is in direct tension with our maxNotes 8->12 bump, which optimized retrieval recall. 12 notes may be past the point where the consuming model loses the middle ones. Retrieval wants recall; generation wants few-and-precise. We optimized only the retrieval side. Re-examine maxNotes; let adaptive-K set it per query.

What actually solves "precision without losing recall" (priority order)

1. Adaptive-K (top recommendation). Replace the fixed cap with a dynamic cutoff from the score distribution / a confidence gap — 3 notes on a focused query, more on a broad one. Solves the exact problem: no force-drop of relevant (it's adaptive), no junk dump (it's thresholded). Plug-and-play, single-pass, no training, runs on the pgvector scores we already compute. (Adaptive-k, Taguchi et al. 2025, arXiv:2506.08479; Cluster-based Adaptive Retrieval, arXiv:2511.14769; CRAG confidence evaluator.)
2. Lost-in-the-middle reordering (cheap). Place the top notes at the start and end of the injected block, not the middle. Trivial; our injection is already an ordered block.
3. Structural signals. Shared-entity / same-project / same-file co-retrieval (Mem0's real lever). We have project-boost; file-path + entity matching are reliable dev-memory cues that cosine misses.
4. Temporal validity, not recency decay. Suppress superseded facts (a reversed decision, a changed preference) — Zep's win. Distinct from recency decay, which we tested and which hurt (dev knowledge is durable). About correctness, not freshness.
5. HyDE as a recall fallback for short/vague prompts, gated on low first-stage confidence only (it adds +25-60% latency + hallucination risk; not always-on). (Gao et al. 2022; medical fine-tune +4.9% NDCG@10, Li et al. 2024.)
6. Cross-encoder: keep OFF. Correct from the eval, now backed by the literature. If ever revisited, fine-tune on our own grade labels rather than a bigger off-the-shelf model.

(c) What production agent-memory systems actually do (2025-2026)

None rank by cross-encoder.

• Mem0: semantic + BM25 fusion (≈ exactly our RRF) + shared-entity co-retrieval.
• Zep/Graphiti: + temporal validity windows + graph traversal + community summaries (beats Mem0 on temporal benchmarks).
• Letta/MemGPT: agent-driven search (the agent decides what to page in), tiered store, no fixed ranker.

The field ranks by structural + temporal signals + LLM/agent selection, not rerankers. External confirmation our energy belongs in (1)-(4).

(d) Evidence caveats (honesty)

Much of the downstream-outcome evidence is industry blogs, not peer-reviewed — treat percentages loosely. Reranking does help some QA tasks; task type is the decisive variable. Lost-in-the-middle is being mitigated by newer order-robust models, shrinking its severity. But the direction — adaptive selection over reranking — is consistent across academic and practitioner sources.

Decision for Argus

Pivot from "make the reranker work" to set selection: ship adaptive-K (biggest, best-evidenced win) + start/end reordering, then structural signals; re-examine maxNotes=12. Leave the reranker built-but-off (ARGUS-85/86 are merged and flag-gated, so they cost nothing idle). Higher-leverage and pgvector-native vs chasing a bigger cross-encoder.

M1 card update (ARGUS-1900/ARGUS-1962/ARGUS-1966, 2026-09-11)

The ARGUS-87 wash above was re-tested at a larger corpus size against a labeled-verdict probe-eval-jobs replay (bge-reranker-base, n=800, ks 5/8/10/12/15/20, checkpoint/resume-backed job 4, clean arm confirmed via rerank_failures: 0, wall-clock 31.64s, status done, no restart during execution — the run completed straight through and never exercised the ARGUS-1941 checkpoint/resume mechanism). Precision@8 off 0.6304 -> on 0.6346 (+0.0042 lift), against the +0.05 gate; recall@20 off 0.98625 -> on 0.98625 (no regression, identical to four decimal places).

This confirms ARGUS-1900's original M1 verdict (precision@8 off 0.6349 -> on 0.6326, -0.0023) even after that number was re-measured through a clean (non-degraded) reranker arm rather than the silent-RRF-fallback path ARGUS-1961 later found and fixed the reporting for. Both the original and the re-run lift sit within noise of zero and nowhere near the +0.05 gate (sampling noise on precision@8 at n=800 is ~0.017, so +0.0042 is a quarter of one standard error) — the sign flipped between runs but the magnitude did not, which is exactly what "no real effect" looks like at this sample size. Decision: recall_rerank_enabled stays OFF in live recall. No config flip made; the gate was not met.

M4 card update (ARGUS-1901, 2026-09-11)

ARGUS-1901 depended on ARGUS-1900's outcome either way (flip the ordering, or document no-lift). The M1 card above already settled that: the reranker gate was not met and recall_rerank_enabled stayed OFF, so the "post-1900 ordering" IS the same RRF-fused order the injection cap was already tuned against in ARGUS-75 — no new ordering to wait on.

GET /recall/probe-eval?ks=5,8,10,12,15,20 (pure SQL over the accumulated recall_probe_verdicts window, no LLM call) against the live corpus:

K	avg precision	avg recall
5	0.7390	0.3357
8	0.7006	0.4934
10	0.6775	0.5868
12	0.6568	0.6755
15	0.6284	0.7987
20	0.5941	0.9931

(5,499 graded prompt-mode injections — every point shares the same denominator, so this is one controlled sweep, not independently-sampled numbers.) These numbers are well above the epic's original framing (precision ~0.40, ~5 relevant notes dropped/query) — that framing predates several since-landed precision wins on this same path (ImportanceWeight ARGUS-89, RecencyHalfLife ARGUS-1902) and is superseded by this measurement, not reconciled with it.

K=20 is the ceiling of what /recall/probe-eval can currently measure: the grade cron's miss-probe re-runs recall at missProbeLimit = 20 notes wide (recall_grade.go), so no judged data exists past rank 20 to sweep further. Recall@20 (0.9931) is effectively complete within that window — going from K=12 to K=20 recovers nearly all of the remaining judged-relevant notes (recall 0.676 -> 0.993) for a graceful, not cliff-edge, precision cost (0.657 -> 0.594), the same shape of trade ARGUS-75's 8->12 bump made on this same lever. This matches the epic's own mandate ("attack capacity (inject more)... with cheap, no-hot-path-cost levers") more directly than AdaptiveKAlpha does here: AdaptiveKAlpha needs a per-note score distribution to cut on, and recall_probe_verdicts stores judged relevance by RANK only (no score column) — there is no accumulated data to calibrate an alpha against, only a flat K is measurable with the tooling that exists today.

Decision: raise the injection cap (promptHookMaxNotes, prompt_hook.go) 12 -> 20. recall_floor (PromptRelevanceFloor, tuning.go) stays at its default 0.015 — probe-eval has no per-note Score column to evaluate a floor change against (see the ARGUS-1901 correction added to that constant's own doc comment: the floor's original RRF-scale calibration basis is stale relative to what HybridRecall's fused Score actually contains today, but fixing that gap needs recall_probe_verdicts to persist Score alongside rank — left as a named follow-up, not built into this change).

Known follow-on caveat, not fixed by this task: missProbeLimit (=20) now equals the new injection cap (=20), so the grade cron's miss-probe window no longer has headroom beyond what is already injected — future probe-eval runs will see probed_count/miss_count trend toward zero going forward even though those columns' PAST accumulated rows (measured against the old 12-cap) remain valid and are exactly what the sweep above used. Re-widening missProbeLimit past the new cap (e.g. to 30) to restore that telemetry is real follow-up work for a future M5, not done here (this task's acceptance is the cap/floor operating point, not the grading cron's own instrumentation).

recall_grade_history accumulates automatically off the grading cron (GradeRecall, internal/memory/internal/store/recall_grade.go) — this PR's own merge/deploy does not itself write a row. The visible discontinuity this change causes (from more, lower-ranked notes now getting graded from rank 13-20 that were dropped by the cap before) will show up as new recall_grade_history rows dated after this change deploys; see the PR's ACCEPTANCE section for how that clause is tracked.

Sources

• Déjean et al. 2024, cross-encoder/LLM/SPLADE comparison — arXiv:2403.10407
• Lost in the middle (Liu et al. 2023) — via arXiv:2512.14313 (Dynamic Context Selection)
• Adaptive-k (Taguchi et al. 2025) — arXiv:2506.08479
• Cluster-based Adaptive Retrieval (CAR) — arXiv:2511.14769
• HyDE (Gao et al. 2022); medical iterative fine-tune (Li et al. 2024)
• Production framework comparisons (Mem0/Zep/Letta) — particula.tech, agentmarketcap.ai, 2026
• CalibRAG; Fin.ai listwise-rerank A/B; CRAG; Self-RAG (practitioner + arXiv)