BLUF: A part number is an identity constraint, not merely a semantic hint. For enterprise RAG over maintenance manuals and product catalogues, keep a dedicated exact-identifier lookup alongside lexical and vector retrieval. Fuse ranked candidate lists rather than adding incompatible raw scores, and preserve verified exact matches through any reranker. Adopt hybrid search only if a query-class evaluation shows that it improves evidence retrieval within your latency budget. A fluent answer about a similar component is still the wrong answer.
Consider the illustrative identifiers AX-104-B and AX-104-8. They are invented examples, not customer records. A technician asks for the tightening procedure for the first component; a semantically similar manual for the second is not an acceptable substitute. Embeddings can help with “replacement seal for the coolant pump,” but their similarity scores do not establish that two catalogue identifiers denote the same physical part. Lexical retrieval is not automatically exact either: tokenisation, punctuation handling and OCR errors can destroy the distinction before ranking begins.
Start with the identity contract, not fusion weights
Store the original identifier, its supplier or catalogue namespace, revision, source document ID and authoritative aliases as structured metadata. Use a dedicated non-tokenised identifier field for equality lookup and a separate analysed text field for descriptive search. Keep the original string for citations. Elastic's keyword-field reference distinguishes structured identifiers and term-level queries from full-text search. The application must know whether a revision suffix changes the supported procedure, whether a distributor code is an alias, and whether case has meaning in that catalogue.
Normalisation is a versioned business rule. Removing hyphens or folding case can improve lookup when the supplier explicitly treats those variants as equivalent. Blindly removing punctuation or converting O to 0 can also collapse different parts into one key. Measure collisions during ingestion; quarantine ambiguous mappings rather than silently choosing a record. Treat OCR substitutions as suggestions that require disambiguation, not as authoritative corrections.
Put the identifier and revision into each relevant chunk's metadata. Include useful title and part context in the embedded passage without assuming that a long document's final appendix will survive truncation. The multilingual E5 model card requires query: and passage: prefixes for asymmetric retrieval, including non-English inputs, and documents a 512-token limit. Those are model-specific requirements, not defaults for every embedder. An indexing manifest should pin the tokenizer, embedding model, normalisation rules and chunking configuration together. The chunking strategy evaluation addresses the evidence-unit decision; this article addresses which evidence units enter the candidate set.
Reference data flow: identity and meaning take different paths
Authenticated request → server-derived permission scope → identifier extraction and ambiguity check → parallel candidate branches → stable-ID deduplication and RRF → exact-match policy → optional reranker → evidence validation → cited answer or clarification.
Branch A: permission scope + namespace + verified identifier + revision → structured equality lookup → exact evidence candidates.
Branch B: the same permission scope + original query + approved aliases → lexical search over identifier-aware fields and descriptive text → ranked lexical candidates.
Branch C: the same permission scope + model-specific query encoding → filtered vector retrieval → ranked semantic candidates.
Fusion receives authorised candidates only. Keep the exact-lookup result as an explicit signal rather than assuming it will win a rank sum. If the user supplied an unambiguous identifier, a safety-oriented policy can restrict answer evidence to that identity and use semantic ranking only within its documents. If multiple revisions match, ask for the revision or show clearly separated alternatives. An exact match identifies a part; it does not prove that a retrieved page contains the requested procedure.
Do not create a privileged fast path for catalogue lookups. Apply the scope to every branch before candidate content leaves the retrieval service. A reranker, parent-document expansion or cache must not widen it. Derive filters on the server, fail closed when scope is unavailable, and include policy revision in cache keys. These controls extend the existing RAG access-control architecture, rather than replacing it with a ranking heuristic.
Decision table: route the query by its failure cost
| Query class | Starting route | Required guard | Escalate when |
|---|---|---|---|
| Verified full identifier | Structured exact lookup, then lexical evidence search | Namespace and revision agreement | No authorised exact evidence exists |
| Identifier with suspected OCR error | Exact lookup plus bounded alias suggestions | Preserve original; do not silently substitute | Multiple distinct parts remain plausible |
| Identifier plus a procedure question | Exact identity constraint plus lexical/vector search within scope | Evidence must support the requested procedure | Correct part, wrong procedure or revision |
| Natural-language symptom | Lexical and vector candidates with RRF | Check compatibility before recommending a part | Several components fit the symptom |
| Catalogue comparison | Retrieve each identity separately, then compare | Separate citations and revision labels | Missing or incompatible source specifications |
This routing policy is an engineering proposal, not a vendor guarantee. A pure exact-lookup service may be sufficient for a clean catalogue with identifier-only queries. A hybrid stack adds an index path, query-time work and failure states. Its value is broader candidate coverage for mixed queries, not permission to guess a replacement part.
RRF combines positions, not confidence
Elastic's reciprocal rank fusion documentation defines a document's fused score as the sum of 1 / (rank_constant + rank) over lists containing that document, with ranks starting at one. Absent documents contribute nothing. This avoids directly adding a BM25 score to a vector similarity score whose scale has a different meaning. Deduplicate by a stable retrieval-unit ID within each list; then merge the same unit across lists. Do not equate a chunk ID with a catalogue part ID: one part can need several complementary evidence chunks.
Microsoft's hybrid ranking documentation explicitly separates the RRF constant from the vector query's nearest-neighbour count. It also places semantic reranking after fusion. These are separate controls: the number of candidates retrieved, the number exposed to fusion, the number sent to the reranker, and the number finally supplied to the generator. Name each independently in configuration and traces.
RRF is not an exact-match guarantee. A generic manual appearing near the top of both lists can outrank an exact-match manual appearing in only one. Nor is an RRF score a calibrated probability that an answer is correct. Adding another vector field or retriever can change the score distribution even when corpus quality is unchanged. Do not reuse an old absolute acceptance threshold after changing branch count. Keep the identity constraint and evidence-sufficiency decision explicit.
Candidate windows can make a reranker irrelevant
Elastic documents that vector results are truncated when k exceeds rank_window_size. If the relevant manual is outside every branch's retained window, neither fusion nor a stronger reranker can recover it. Diagnose the first point of loss: source missing from index, identifier mismatch, branch omission, candidate truncation, fusion demotion, reranker demotion, or prompt-context truncation.
Start with a bounded sweep of branch windows on a frozen evaluation set. Record candidate-union recall before fusion as well as recall at the final context cutoff. Increase one window at a time, keeping the embedder, corpus, filters and other settings fixed. Wider windows may improve recall, but they increase downstream sorting, reranking and context-selection work. The useful configuration is the smallest one meeting the agreed segment floors, not the largest one the service accepts.
For parallel branches, record individual branch duration, total retrieval wall time, reranker time and end-to-end answer latency. A slow branch can dominate tail latency; measuring only fusion time hides it. Set timeouts and define a labelled degraded mode. For identifier-bound requests, a vector timeout may permit an exact/lexical-only answer if authorised evidence is sufficient. An exact-lookup outage must not silently become an approximate-match answer. An authorization outage always denies retrieval.
Ablation scorecard: collect measurements, do not invent a winner
Use a versioned query set with separate full-identifier, spelling/OCR-variant and natural-language classes. Record relevant document and evidence-span IDs, acceptable revisions, permission scope, query provenance and expected abstention. Add hard negatives differing by one character, the same code in another supplier namespace, obsolete manuals and unauthorised near-duplicates. Keep development examples separate from the held-out release set.
The following table is an evaluation worksheet, not benchmark results. NM means not measured: no production corpus, retrieval service or representative load test was supplied for this article. Each recall cell requires its own query count and uncertainty assessment. Repeat the rows with and without the exact-match routing policy to isolate its contribution; otherwise a “hybrid win” may actually be an identity-routing win.
| Variant | Exact recall@10 | Variant recall@10 | Natural-language recall@10 | End-to-end p95 |
|---|---|---|---|---|
| Lexical only | NM | NM | NM | NM |
| Vector only | NM | NM | NM | NM |
| Lexical + vector, RRF | NM | NM | NM | NM |
| RRF + reranker | NM | NM | NM | NM |
| RRF with wider candidate window | NM | NM | NM | NM |
Define recall@10 as the fraction of annotated relevant evidence units present in the first ten retrieved units, averaged within each query class. Also report candidate-union recall, MRR for the first useful result, and nDCG when graded relevance labels exist. Score part/revision correctness and answer support separately. No-answer queries have no recall denominator; evaluate correct abstention and false substitution instead of forcing them into the recall average.
For latency, declare hardware or service tier, corpus size, concurrency, query count, warm/cold-cache handling and whether answer generation is included. This worksheet asks for end-to-end p95, so retain branch and retrieval-only timings as diagnostics. Bootstrap or otherwise quantify uncertainty over queries where appropriate; a handful of examples is a debugging set, not reliable evidence of a general quality improvement. Fix acceptance floors and tolerable regression before tuning, then reject a candidate that trades exact-identity failures for a better blended average.
Release checklist and operational evidence
Ingestion check: detect normalisation collisions; validate namespace and revision metadata; confirm identifiers survive OCR and extraction; verify document deletion and supersession.
Retrieval check: prove identical permission scope on exact, lexical and vector branches; log branch availability, candidate counts, stable IDs and the stage at which annotated evidence disappears. Avoid retaining raw sensitive queries or document bodies in routine telemetry.
Ranking check: preserve an explicit exact-identity decision; verify stable tie-breaking; test deduplication; compare with and without reranking and wider windows. A reranker must not replace the identity contract.
Answer check: verify citations support the procedure for the requested part and revision. Refuse or clarify when identity or evidence is ambiguous. Search relevance is not engineering approval to install a substitute component.
Rollout check: shadow the candidate against the frozen baseline, review failures by query class, then canary with rollback to the prior index and retrieval manifest. Monitor zero-result rate, clarification rate, false substitutions, timeouts and cost alongside user-visible latency. Changes to aliases, embeddings or candidate windows require a fresh regression run.
What to build first
Take one bounded catalogue and its manuals. Establish exact-lookup correctness, then add lexical and vector candidates only where real question classes need them. The first deliverable should be an identity contract, a labelled evidence set and an ablation report—not a claim that hybrid retrieval is universally superior.
If your assistant returns plausible manuals for the wrong parts, I can help review the identifier schema, retrieval traces and evaluation design. Bring a representative corpus slice, failed queries and your latency and access constraints. The decision to make is whether fusion improves authorised evidence coverage without weakening exact identity—not whether the architecture contains a vector database.
