Summary
similar_resolved_incidents applies a status IN ('resolved','closed') filter after the ANN traversal with a hard LIMIT k. pgvector's HNSW filters post-traversal, so with the default ef_search=40 and a corpus dominated by open incidents, the query can silently return fewer than k results or miss closer resolved matches.
Evidence
sentinel/persistence/repositories.py:882-931 — ORDER BY i.embedding <=> CAST(:qvec AS vector) LIMIT :k (:908-909) with WHERE ... i.status IN ('resolved','closed') (:903-907).
- No over-fetch multiplier (
LIMIT :k is bound directly), no ef_search tuning, no partial index on resolved rows (the only HNSW index idx_incidents_embedding is unfiltered), no hnsw.iterative_scan.
- The EXPLAIN test only validates the unfiltered
ORDER BY.
Fix (options)
- Over-fetch then filter:
LIMIT k * N in a subquery, filter to resolved/closed, take top-k.
- Or raise
ef_search for this query (SET LOCAL hnsw.ef_search).
- Or add a partial HNSW index on resolved/closed rows.
- Or enable
hnsw.iterative_scan (pgvector ≥ 0.8).
- Add a recall test that seeds a resolved-sparse corpus and asserts k results.
Summary
similar_resolved_incidentsapplies astatus IN ('resolved','closed')filter after the ANN traversal with a hardLIMIT k. pgvector's HNSW filters post-traversal, so with the defaultef_search=40and a corpus dominated by open incidents, the query can silently return fewer thankresults or miss closer resolved matches.Evidence
sentinel/persistence/repositories.py:882-931—ORDER BY i.embedding <=> CAST(:qvec AS vector) LIMIT :k(:908-909) withWHERE ... i.status IN ('resolved','closed')(:903-907).LIMIT :kis bound directly), noef_searchtuning, no partial index on resolved rows (the only HNSW indexidx_incidents_embeddingis unfiltered), nohnsw.iterative_scan.ORDER BY.Fix (options)
LIMIT k * Nin a subquery, filter to resolved/closed, take top-k.ef_searchfor this query (SET LOCAL hnsw.ef_search).hnsw.iterative_scan(pgvector ≥ 0.8).