Filtered Vector Search Recall: Compare Approximate Results with an Exact Scoped Baseline

A filtered vector query that returns no neighbours does not prove that the permitted corpus is empty. Compare approximate search with an exact top-k under the same mandatory predicate, distance metric and database snapshot before changing embeddings. In the synthetic database fixture executed for this article, one tenant had 200 eligible vectors: HNSW with iterative scans disabled returned none, while the exact query returned ten. Iterative scanning recovered those ten, but took more work. Never remove the tenant filter to repair recall.

Separate three different failures

First ask whether the source exists and is visible to the caller. Then ask whether approximate search finds its nearest neighbours. Only then ask whether those neighbours contain relevant evidence. Nearest-neighbour recall measures agreement with the vector distance ranking; it does not measure factual correctness, human relevance or answer quality.

The multi-tenant RAG isolation design establishes the authorization boundary. This article diagnoses candidate loss inside an unchanged scope, not whether that scope is secure. The hybrid retrieval guide addresses exact part identifiers and relevance across lexical and semantic retrieval. The reranker token-budget diagnostic addresses evidence discarded after candidates already exist. Rebuilding all three layers at once makes the cause harder to identify.

Why a filter can leave HNSW short

The pgvector reference distinguishes exact search from approximate indexes and documents that filtering with an approximate index is applied after its index scan. A bounded candidate search may encounter mostly rows outside the required scope. An existing HNSW index therefore does not promise k surviving rows, and increasing k does not automatically increase the useful search budget sufficiently.

This is an execution-order observation, not permission to expose unauthorized candidates. The application must receive only authorized results. Bind the tenant from trusted identity, preserve every authorization predicate in both diagnostic queries, and keep restricted identifiers out of ordinary logs. Validate row-level security with the production application role separately; a superuser fixture cannot prove it.

Since pgvector 0.8.0, iterative index scans can continue searching until enough results are found or their configured limits are reached. For HNSW, strict_order preserves distance ordering of the returned results. It does not turn an approximate graph search into exhaustive exact search or guarantee recall of one. Treat ef_search, max_scan_tuples and scan_mem_multiplier as bounded experiment parameters, not universal tuning recommendations.

The executed fixture and its limits

The local run used PostgreSQL 18.6 and pgvector 0.8.1, Python 3.12.13 and psycopg 3.3.6. It inserted 20,000 synthetic, 16-dimensional vectors, generated with Python Random(71), uniform coordinates between minus one and one, rounded to seven decimal places. Tenant populations were deliberately skewed: 10,000, 2,000, 200 and five rows, with the remainder in a fifth tenant. One additional nonexistent tenant tested empty scope. A single subsequent random vector was the query; distance was L2 and k was ten.

The HNSW index used m=16 and ef_construction=64 with serial construction. Comparisons ran in one REPEATABLE READ transaction. Exact measurements disabled index and bitmap scans; ANN measurements disabled sequential and bitmap scans and asserted that fixture_hnsw actually executed. These settings isolate the mechanisms for a diagnostic; do not copy forced planner settings into production. Prepared statements were disabled in the diagnostic client so that plan reuse did not invalidate the intended per-route comparison.

Resource settings were shared_buffers=128MB, work_mem=4MB, maintenance_work_mem=128MB for construction, no parallel query workers, JIT disabled and a ten-second statement timeout. Iterative scans used max_scan_tuples=20000 and scan_mem_multiplier=1. Those limits are not measurements of total process memory or a guarantee of visiting every vector. The test had one warm-up and seven timed executions per route; values below are median client round trips on a local Unix socket, including planning, not production tail latency. Query plans with EXPLAIN ANALYZE and buffer counters were retained separately.

Each cell reports returned rows, exact-neighbour overlap and median milliseconds. E means forced exact scan; A40 and A200 mean HNSW with ef_search 40 or 200 and iterative scanning off; I40 means ef_search 40 with strict_order iterative scanning. The denominator is min(k, eligible rows), not always ten. Empty scope has undefined recall, not a perfect or failed recall score.

At 1% eligible population, simply increasing ef_search to 200 still returned zero in this run. I40 recovered ten out of ten at 6.752 ms; exact took 1.439 ms. For only five eligible rows, iterative scanning returned all five but took 61.997 ms versus 1.186 ms for exact. These are observations from one small synthetic query and one graph build. They are not evidence that exact search is generally faster, or that these parameters work for production embeddings.

Make the baseline genuinely exact and scoped

Use one parameterized statement for both routes: SELECT id, embedding <-> %s::vector AS distance FROM recall_fixture WHERE tenant=%s ORDER BY embedding <-> %s::vector LIMIT 10. Bind the identical query vector in both vector positions and the same trusted tenant value. The fixture connection used prepare_threshold=None. Preserve the model version, normalization contract and metric in the replay manifest; switching cosine to L2 is not an ANN-only comparison.

Begin a REPEATABLE READ transaction before counting the eligible population and executing either route. For an isolated diagnostic without metadata indexes, SET LOCAL enable_indexscan=off and SET LOCAL enable_bitmapscan=off produced Limit → Sort → Seq Scan for the exact route. Restore index scans, disable sequential scans only for the controlled ANN experiment, set the HNSW budgets, and rerun the identical SELECT. Inspect EXPLAIN (ANALYZE, BUFFERS, FORMAT JSON) for each setting. EXPLAIN ANALYZE executes the query; budget that load rather than treating it as a free inspection.

A plan from a different connection or a separately reconstructed statement is insufficient proof. Parameter types, prepared plans, transaction state and planner settings can change the path. Our first diagnostic attempt failed an assertion after automatic statement preparation; the retained successful fixture explicitly disables it. Production prepared queries should instead be tested in their actual prepared form, with representative parameter distributions.

Compute neighbour recall as the number of distinct returned IDs also present in the exact set, divided by the exact set size. Report returned-row count separately: ten approximate rows can still be the wrong ten. Inspect ties at the cutoff before penalizing alternative equally distant IDs. A separate executed two-vector check placed [1,0] and [-1,0] at equal L2 distance from [0,0]; an arbitrary ID tie-break would choose one without establishing superior relevance. Use a documented tie-aware rule or compare the boundary distance group. Do not add an ID sort to the ANN query without verifying that the intended index path survives.

Let the planner choose when the mechanism is understood

A second executed control added a B-tree on tenant and restored ordinary scan choices. For the five-row tenant, the observed plan was Limit → Sort → Index Scan on fixture_tenant, not HNSW and not an assumed combined B-tree/HNSW path. Exact distance sorting after a selective metadata lookup can be a useful alternative. Measure it with your data, statistics and concurrency.

PostgreSQL's multicolumn-index documentation explains access-method-specific behavior. It does not establish that you can create a multicolumn HNSW index combining a tenant key and a vector, or that PostgreSQL will combine two existing indexes for your ordered query. Use the EXPLAIN documentation to inspect actual rows, filtering and scan choices rather than treating an index definition as execution evidence.

A release checklist beyond this small experiment

Freeze a representative query set, corpus revision, caller scopes and encoding contract. Include broad and narrow tenants, empty scopes, fewer-than-k populations, distance ties, skewed metadata, deletions and real permission restrictions. Use multiple queries and repeated builds; our uniform synthetic vectors do not reproduce semantic clustering or production dimensions.

Measure the ordinary planner route alongside each controlled route. Record exact-neighbour recall, returned rows, latency distributions, timeout rate and resource use under realistic concurrency. Keep query warm-up, cache state and diagnostic EXPLAIN overhead explicit. Establish workload-specific recall and latency thresholds before selecting a configuration; do not promote the largest search budget by default.

Require independent cross-tenant negative tests under the real application role, including missing context, changed permissions and caches. This fixture uses explicit predicates but has no RLS policy. Its successful recall checks do not certify authorization. Do not convert an empty result into an unfiltered retry, and do not log restricted chunk bodies to explain the miss.

Choose the next experiment, not a new model by default

If exact scoped search also lacks the expected evidence, inspect ingestion, eligibility, the embedding contract and labelled relevance. If exact succeeds while ANN fails, test bounded search work or an exact filtered route first. If candidates survive retrieval but vanish later, inspect reranking and prompt assembly instead.

For a retrieval reliability review, bring one reproducible failed query, the authorized scope, index definition, actual plan and latency budget. I can help turn those into a scoped evaluation and a release decision. The useful outcome is evidence about where candidates disappear—not a higher result count obtained by weakening access control.