Conflicting Manuals in RAG: Resolve Applicable Revisions Before Ranking Evidence

Resolve which document revision applies before ranking passages. A maintenance answer can quote a real, accessible manual and still be wrong for the asset configuration, region or effective time. Use a governed applicability registry to produce an authorised revision set, carry that scope through retrieval and citation, and abstain when authority remains unresolved. Neither a newer filename nor a higher similarity score establishes the correct instruction.

This article proposes a bounded implementation and executes a synthetic decision fixture. It does not report a customer deployment or validate maintenance instructions. The torque values below are invented test data and must not be used on equipment. The implementation question is whether the retrieval system can keep a plausible but inapplicable instruction out of the answer.

The conflict is not a ranking problem

Suppose the pump-seal family for configuration C2 in the EU contains manual A with 40 Nm, bulletin B with 45 Nm and replacement manual C with 50 Nm. A covers logical time [0,10), B covers [5,10), and C covers [10,infinity). B explicitly supersedes A for the same procedure and scope. A German translation is equivalent only to A. These are logical test ticks, not dates or real product specifications.

At tick 4, A applies. At tick 6, B wins because a reviewed supersession relationship exists—not because its wording looks more recent. At tick 10, C applies and the prior intervals exclude that instant. An accessible German A cannot support a current C instruction merely because it is in the requested language. If a second approved revision overlaps C without a resolved authority relationship, the correct result is a conflict, not a reranker tie-break.

Permissions and applicability are separate decisions. An employee may be allowed to read all historical manuals without being allowed to use any of them for today's assembly. Conversely, if the applicable bulletin is inaccessible, returning an older permitted manual is not an acceptable fallback. The foundational RAG data-governance controls establish the access boundary; the applicability resolver must preserve it without mistaking accessibility for authority.

Define the registry contract before indexing

Store document_family, immutable revision_id, source_digest, procedure_id, asset_configuration and region as explicit fields. Include valid_from and valid_to, source_language, reviewed equivalence links, authority decision reference, and supersedes relationships scoped to the affected procedure. Record ingestion time separately: a late import of an old manual does not make it current. A PDF's modification timestamp is not an engineering release decision.

Separate business valid time from knowledge or transaction time. “Which instruction applies to the asset as of tick 6?” can mean the historical valid-time answer using today's corrected registry. “What could the assistant have known at tick 6?” requires a historical registry snapshot or bitemporal history as well. Do not promise the second question when you retain only the latest metadata. Keep revision contents immutable; correct their registry interpretation through versioned decisions rather than replacing source bytes under the same ID.

Use timezone-aware instants for real timestamp-based rules and define how date-only supplier releases become plant-local effective instants. Prefer half-open intervals: valid_from <= as_of and (valid_to is absent or as_of < valid_to). Validate non-empty intervals and decide whether gaps are legitimate. A missing end can mean unbounded only under an explicit schema; missing or malformed metadata must not silently acquire the same meaning.

Treat authority as reviewed domain data. A bulletin may override one procedure while leaving the rest of a manual intact. Resolve at the smallest governed authority unit, such as family plus procedure and configuration, not by globally deleting the superseded manual. Validate supersession edges for matching scope, effective intervals, cycles and missing references. The small fixture here assumes approved direct edges; a production registry needs those ingestion checks and a documented rule for transitive chains.

Bind each request to an applicability decision

Resolve tenant and current authorisation first. Then obtain a verified asset configuration, procedure, region and question mode. For current-use questions, capture the request's effective instant once. For historical questions, require the explicit as-of instant and label the result as historical. A guessed asset generation is not a retrieval parameter: ask for clarification or route to an approved manual lookup process.

Inside a trusted service, consult enough authority metadata to detect that an inaccessible revision supersedes a visible one. That metadata is not returned to the caller or sent to a model. If the service cannot establish authoritative completeness without exposing restricted information, fail closed with a generic unavailable result. Otherwise an attacker—or an ordinary permission change—can turn access filtering into obsolete-instruction selection.

Create a scope token containing tenant, authorisation epoch, asset configuration, region, procedure, current/historical mode, resolved instant, registry snapshot, applicability-policy version and allowed revision IDs. Attach that token to every lexical, vector and reranking path. Return only chunks whose immutable source revision belongs to that set. Before generation, verify every selected evidence ID against the scope; at citation time verify it again. Fail closed on missing metadata or mismatched revisions.

The diagram is the synthetic valid-time example, not a real maintenance schedule. The exclusive boundary at tick 10 prevents A or B from coexisting with C solely because of an inclusive end. The overlap between A and B is deliberate: the authoritative supersession decision resolves it. Without that decision, overlap is an unresolved conflict.

Decision table: preserve the meaning of the question

An approved source-language workflow can generate a German explanation from an applicable English source, but that is not proof that a German supplier edition exists. Keep source language, answer language and equivalence status distinct. The cross-language retrieval evaluation addresses retrieval quality after revision equivalence is fixed; it cannot manufacture that equivalence.

Use filters to enforce the result, not to invent authority

Qdrant's filtering documentation defines Boolean filter combinations and datetime range conditions. Use those primitives to intersect the authorised resolved revision set with the tenant and other required scope fields. If applicability is represented as an array of configuration/region objects, use the documented nested-object semantics so matching values must belong to the same object. Independent array matches can accidentally combine a configuration from one entry with a region from another.

Azure AI Search's OData filter reference likewise provides inclusion/exclusion expressions, comparisons and Boolean operators. A conceptual predicate is tenant equals the authorised tenant AND revision belongs to the resolved set. Construct it with a tested encoder and typed values, not string interpolation from the user's question. Verify the deployment's filterable field schema, vector-filter execution mode and empty-set behaviour; the OData syntax reference alone does not establish vector-search recall or prefilter configuration.

For a revision registry, PostgreSQL range types support inclusive/exclusive bounds, overlap tests and exclusion constraints. An optional canonical applicability table can use a timestamp-with-time-zone range and an exclusion constraint over scope equality plus interval overlap, using btree_gist for compatible scalar keys. Scope keys must be non-null and normalised. This constrains accepted canonical decisions; it does not decide which competing supplier instruction deserves acceptance.

Do not add a blanket no-overlap constraint to the raw source table if bulletins legitimately overlap base manuals. Keep raw evidence and unresolved conflicts there; build a reviewed canonical projection that slices intervals and selects one authority per governed procedure. Wildcards, configuration hierarchies and region inheritance require their own normalisation or conflict checks. A scalar equality constraint cannot detect every semantic overlap. These database and engine patterns were researched, not integration-benchmarked for this article.

Execute the negative cases before tuning similarity

The accompanying local Python fixture ran 21 assertions successfully. It models four immutable records: A, B, C and the German A translation. Thirteen resolver cases cover effective boundaries, current and historical instants, missing scope, wrong region, stale translation, permission-blocked supersession and an unresolved competing revision. Eight cache checks verify that changing policy, resolved revisions, authorisation epoch, registry snapshot, locale, instant, mode or tenant changes the key. These are deterministic application tests, not a retrieval-accuracy score or an engineering safety certification.

The resolver's essential sequence is: validate scope; select approved valid-time candidates; resolve explicit supersession in trusted metadata; require exactly one canonical winner; require permission for that winner; then select a reviewed language-equivalent edition. The fixture intentionally refuses an older accessible document when its successor is forbidden. It also refuses a German edition with no equivalence to the winning revision. A production implementation must additionally validate the registry, evaluate the deployed engine, and test concurrent policy updates.

Reproduce the core cases with logical instants 4, 5, 9, 10 and 12. Expect A, B, B, C and C respectively. Add a second unsuperseded approved record at [10,infinity): expect UNRESOLVED_AUTHORITY. At tick 6 allow only A and its translation: expect UNAVAILABLE, not A. At tick 12 request German against the supplied corpus: expect NO_EQUIVALENT_EDITION. Delete the configuration: expect MISSING_SCOPE before any vector query. Turn these expectations into CI fixtures owned jointly by retrieval engineering and the document authority owner.

Then test the actual index: put obsolete and current passages with near-identical wording into it, force the obsolete passage to rank highly, and assert that it never reaches the reranker or answer context for current use. Repeat with lexical retrieval, query expansion and cached answers. This integration test remains a required next step; the local resolver run does not claim it has exercised your Qdrant or Azure deployment.

Make citations and caches carry the decision

A citation manifest should include request-scope ID, revision ID, source digest, procedure/section, chunk ID, source language, reviewed equivalence ID when used, effective interval, resolved instant, registry snapshot, policy version and access-decision reference. For the synthetic tick-6 answer, the manifest identifies bulletin-B, pump-seal/C2/EU, interval [5,10), source language en and the pinned registry/policy versions. Its 45 Nm value remains test data, not an operating recommendation. Keep sensitive authority and access details in restricted logs; expose only what the caller may see.

Cache by the resolved decision, not just normalised question text. Include tenant, relevant access context, policy and registry versions, resolved revision set, language, configuration, region, procedure and historical/current mode. A literal per-request timestamp key is easy to reason about but yields poor reuse. If you optimise with a validity window, prove that no authority boundary can be crossed during that window and invalidate on corrections or revocation. Reauthorise on read where required. Never let an old answer cache bypass the resolver.

Pin the scope for request consistency, then define what happens if an urgent revocation arrives while generation runs. High-impact workflows may require a final freshness check and restart or refusal. A pinned snapshot is reproducible but can still be too stale to serve. During embedding index migrations, keep this applicability contract separate from the encoder/index route and enforce both; compatibility of vector spaces says nothing about instruction validity.

Release checklist, cost and a bounded next step

Registry gate: immutable content digests, reviewed authority edges, complete scope keys, explicit time semantics and monitored unresolved conflicts. Retrieval gate: every route enforces the resolved revision set; unknown payloads fail closed; empty applicability does not trigger an unrestricted search. Evidence gate: citations retain scope and revision identity; translations cannot silently substitute older instructions. Operations gate: caches respect revocation, registry corrections and effective boundaries; an owner can reconcile every blocked case.

The trade-off is additional metadata ownership, resolver latency and more abstentions when source quality is poor. A cached canonical registry projection can reduce query work, but introduces freshness and invalidation responsibilities. A strict gate can reveal that the document estate does not contain an applicable approved instruction. That is an operational finding to resolve, not a reason to relax the filter or ask an LLM to adjudicate engineering authority.

Start with one manual family and one procedure whose revisions genuinely conflict. Bring the asset-configuration rules, authoritative bulletins, language editions and current access policy. I can help your team turn that material into an applicability registry, adversarial fixtures and a release checklist before expanding the assistant. Keep maintenance approval and engineering responsibility with the qualified domain owners; a well-cited generated answer is not permission to execute work.