An embedding migration is a change to a coupled retrieval system, not a vector-copy job. Build a separate destination index, regenerate embeddings from authoritative text, replay updates and deletions, and promote the query encoder and its index as one immutable route. Keep the old route current enough to roll back. An alias swap alone cannot coordinate model services, ingestion workers, caches and permissions.
The dangerous failure is not always a dimension error. Two models can emit vectors of the same length while representing text differently. A query encoded by the new model against old document vectors can pass a shape check and still retrieve the wrong evidence. This article proposes an implementation and acceptance contract; it does not report a customer migration or promise zero downtime.
Define the unit of compatibility
Create a route manifest with route_id, query_encoder_revision, document_encoder_revision, tokenizer_revision, query_prefix, document_prefix, pooling, normalization, vector_dimensions, similarity_metric, chunker_revision, physical_index and authorization_policy_revision. Query and document encoders must form the documented compatible pair; they need not be identical architectures in every retrieval system. Resolve immutable model revisions rather than floating names.
Bind a request to that manifest once, before embedding. Pass the same route ID to the encoder, retriever and telemetry. Validate the resulting embedding's contract ID as well as its dimension before searching. Do not read a mutable global index alias after producing a vector: an in-flight request could observe a different destination than the one it was encoded for. During rolling deployment, old and new requests may coexist safely if each remains inside its own complete route.
This makes the foundational LLM release bundle concrete for retrieval. The release artifact is immutable; the indexed documents remain a changing materialized view. A route manifest therefore needs a separate ingestion checkpoint and deletion ledger, not a misleading claim that pinning configuration freezes source data.
Re-embed text; do not rename old vectors
Elastic's reindex API documentation says that reindex copies documents and requires _source. It does not copy index settings, mappings, shard counts or replicas. Configure the destination explicitly. A plain copy of a stored vector field is not inference with a new embedding model. A purpose-built ingest inference pipeline can regenerate embeddings, but only when explicitly configured and tested; reindex by itself does not establish that contract.
Use the authoritative parsed text and source revision, not an opaque vector export. If text exists only as truncated chunks in the old index, first determine whether those chunks preserve the new model's required evidence. Freeze chunking for the initial migration comparison where possible. If a new token limit forces rechunking, label the release as a combined change and retain a document-to-chunk manifest so obsolete chunks can be removed.
For example, the multilingual E5 large model card specifies 1,024-dimensional embeddings, query: and passage: prefixes for asymmetric retrieval even for non-English text, and a 512-token limit. Its examples use attention-mask-aware mean pooling and normalization. These are that model's requirements, not universal embedding defaults. Include prefixes, titles and special tokens in length checks. Do not silently truncate a safety clause and call the backfill successful.
Validate dimensions and finite values before writes, and verify the chosen similarity and normalization contract against the deployed vector engine. Record model revision, source content digest, chunk identity and encoding-contract ID alongside each output. Treat mismatched or unembeddable records as quarantined work with an owner; document counts alone cannot prove migration completeness.
Build two projections from one durable change stream
The proposed architecture below separates reconstruction from live change capture. The shared source log carries document updates, ACL changes and deletions. It does not carry one model's vectors into the other model's index. Both encoders consume text under their own contracts.
Record a source checkpoint before the snapshot and preserve every later change. For a partitioned stream, this is usually a set of per-partition positions, not one global integer. The snapshot and capture mechanism must cover every change without a gap; overlap is acceptable when replay is idempotent. Use the source system's documented snapshot/CDC procedure. If you cannot establish that boundary, use a maintenance window or reconciliation that demonstrably covers the gap rather than claiming an online guarantee.
Run independent old-route and new-route consumers from a transactional outbox or other durable source feed. Best-effort dual HTTP writes are insufficient: the first write can succeed and the second fail. Persist per-target acknowledgement, retry state and a high-water mark for contiguous successfully applied events. The highest offset ever observed is not a completeness watermark if earlier events remain unresolved.
Use monotonic source revisions within each document's ordering scope. A slower backfill must not overwrite a newer update. A delete must leave a revision fence or durable tombstone so a late embedding result cannot resurrect the document. When an update changes chunk boundaries, reconcile the complete new chunk set and remove superseded chunks. Prevent stale workers from inserting obsolete chunk IDs after that reconciliation. These are application requirements; they are not guarantees supplied by an alias.
Elastic documents that default internal-version reindexing can overwrite matching destination IDs, while external versioning preserves source versions and only updates older destinations. That mechanism is useful only when the version has the right authoritative meaning. It does not invent a consistent source revision across independent systems or replace a durable deletion ledger. Test your actual write and delete semantics, including retries after tombstone expiry.
Make promotion a decision, not a timestamp
| State | Required evidence | Operator decision | Stop condition |
|---|---|---|---|
| Backfill | Snapshot coverage; encoding failures accounted for | Continue serving old route | Missing source range or unknown chunk ownership |
| Catch-up | Every partition applied through barrier; deletes and ACL changes reconciled | Allow controlled shadow traffic | Unresolved event before barrier |
| Shadow | Authorised evidence comparison; latency and cost measured | Approve bounded canary | Critical quality or permission regression |
| Canary | Complete route selected per request; both projections maintained | Expand traffic deliberately | Mixed encoder/index contract |
| Promoted | New route serves traffic; old route passes freshness checks | Keep rollback window open | Old route misses required changes |
| Retired | Rollback window closed; retention and deletion work approved | Remove old serving path and credentials | Legal hold or deletion obligations unresolved |
A catch-up barrier establishes that the candidate has applied a defined source prefix. It does not make every subsequent source write synchronously searchable. Define a permitted ongoing ingestion lag and search-visibility check for both routes. Where the service promises stronger read-after-write behavior, coordinate a write barrier or brief write pause with the application owner. Do not use a checkpoint to imply a consistency level the product does not provide.
Shadow traffic is still production data access
Run the new route with the same authenticated tenant and subject scope as the live request. Apply authorization before candidate retrieval and recheck document access where the policy requires it. A shadow response not shown to the user can still leak restricted snippets into traces, external encoders or evaluation stores. Use minimal evidence identifiers, access-controlled logs and approved data-processing boundaries.
The existing multi-tenant RAG isolation controls apply to both indexes and every cache. Version cache keys by route, tenant and relevant authorization context; invalidate or reauthorize cached results after permission changes. A migration must not revive an old answer cache containing deleted content.
Compare both routes on frozen, permission-aware fixtures: relevant evidence coverage, no-answer behavior, obsolete-document exclusion and critical language/domain slices. Compare rankings or labelled relevance, not raw similarity scores across models. Measure p95 latency, errors, ingestion lag and cost per attempted request under the intended concurrency. The RAG evaluation harness supplies the wider quality gate; this migration adds state consistency and routing tests. A higher average retrieval score does not waive a deletion or tenant-isolation failure.
Cut over the route, not just the alias
Elastic's alias documentation supports remove/add actions in a single atomic alias operation. It also documents partial action failures, including an acknowledged response with errors: true. Inspect action_results and use must_exist where appropriate to make missing-target preconditions fail rather than silently creating an unintended mapping. Verify the actual alias state after the operation, including when the caller times out.
Even a successful atomic alias operation covers Elasticsearch's alias state, not a separate encoder deployment. Prefer immutable physical-index references inside route manifests, with one controlled pointer selecting the entire route at request entry. Canary assignments and in-flight requests retain their selected route. If an alias must change, give it route-specific ownership and prevent old encoder requests from following a newly redirected target.
Use a compare-and-swap deployment decision against the expected active route. Verify candidate readiness first, then publish the pointer and observe which version each worker actually serves. Mixed application versions are acceptable only if each understands the route contract or fails closed. A fleet-wide simultaneous switch is not required; request-level compatibility is.
Rehearse failure before approving rollback
Test 1 — wrong dimension: inject an embedding with an incompatible length. Expect rejection before the vector query and a contract-mismatch event, not silent padding or truncation.
Test 2 — same dimension, wrong encoder: label the vector with the new encoding-contract ID while targeting the old route. Expect rejection despite a valid array shape. A dimension-only guard must fail this test.
Test 3 — stale delete: schedule embedding for document revision A, process its later deletion, then deliver the delayed embedding result. Expect no searchable document in either projection. Repeat after a worker restart and with changed chunk boundaries.
Test 4 — mixed route during deployment: pause a request after encoding, promote the candidate, then resume. Expect the request to query its originally pinned index, not whichever alias now happens to be active.
Test 5 — unauthorized shadow: run a tenant A query with a tenant B canary document in both indexes. Expect no B identifiers or text in results, reranker input or diagnostic logs. Repeat after an ACL revocation and cache hit.
Test 6 — stale rollback target: stop the old consumer, change a document and delete another, then request rollback. Expect the controller to refuse until catch-up and search visibility are verified, or select an explicitly approved degraded mode that cannot reveal stale restricted content.
An old index that still exists is not automatically a safe rollback target. Keep its encoder artifacts, credentials, write consumer and policy handling operable for the agreed window. Continue deletions on retained copies. If compatibility, freshness or permissions cannot be restored quickly, prefer a bounded service interruption to serving forbidden evidence. Retire old routes deliberately once rollback value no longer justifies their cost and retention burden.
Costs, limits and a practical starting point
Parallel indexes increase storage and indexing work; shadow queries add inference and search load; independent consumers increase operational state. The capacity increase is workload-dependent, not automatically an exact doubling. Test backfill throttling against live latency and reserve retry capacity. A migration runbook cannot repair a poor relevance model, incomplete source capture or an authorization architecture with bypasses.
Start with one representative corpus partition. Define the route manifest, a gap-free source boundary, delete fencing and the promotion table before launching a full re-embedding job. Rehearse a delayed update, a delayed delete and an in-flight request across cutover. If your team needs a migration review, bring the source-change contract, encoder configuration and current rollback runbook: I can help turn those into executable acceptance tests and a cutover decision with explicit failure boundaries.
