BLUF — Share a base model only after you have bounded adapter churn, not merely counted adapter files. Multi-LoRA serving can avoid duplicating the base weights across task-specific variants, but the capacity decision depends on which adapters arrive together, which weights are resident and whether every request executes the approved revision. A warm, popular adapter can hide cold-load failures for the long tail. Test identity correctness and per-adapter latency before consolidating production pools.
This is an implementation and experiment design, not a GPU benchmark. No throughput increase, memory saving percentage or production capacity is claimed here. The decision is narrower than the existing quantisation and adapter quality gates: assume each variant has passed its quality evaluation, then determine whether those approved variants can coexist operationally.
Three controls, three different limits
vLLM's stable LoRA guide documents adapter selection through the request's model field and concurrent processing subject to server configuration. Multi-LoRA means different requests can use different adapters; it does not mean arbitrary adapters are combined into one request. Confirm that your exact base architecture and adapter structure are supported by the deployed engine.
The LoRAConfig reference defines max_loras as the maximum number of LoRAs in a single batch. It is not a tenant quota, a request-concurrency limit or a promise that all registered adapters remain warm. max_cpu_loras bounds adapters stored in CPU memory and must be at least max_loras. max_lora_rank bounds supported rank; raising it unnecessarily can increase allocated memory. Do not treat those three controls as interchangeable cache sizes.
Pin the runtime image digest and retain its serve --help output alongside the manifest. The queue's original research referenced developer-preview documentation; this article checks the stable documentation too, but even a stable URL is moving documentation rather than proof about your installed binary. Verify accepted values and loading behaviour against that binary before using the experiment.
Separate adapter identity from customer input
The serving gateway should map an authenticated workload and an approved release to an immutable adapter alias. It must not forward an arbitrary customer-supplied model name, file path or repository identifier. Use an allowlist keyed by workload identity, with the base revision, tokenizer/template revision, adapter digest, rank, target modules, runtime digest and approval attached. Bind this to the LLM release bundle.
A conceptual route is workload extraction-de → release extract-r7 → base digest plus adapter digest → serving alias extract-r7. Resolve the route once at admission and retain that revision through completion. An unknown alias, missing artifact or unauthorized workload must produce an explicit rejection; never silently fall back to the base model. A syntactically successful response from the wrong adapter is still a failed request.
Keep runtime loading disabled for the production experiment: leave VLLM_ALLOW_RUNTIME_LORA_UPDATING unset, verify inherited environment settings and do not enable a dynamic resolver. vLLM warns that runtime loading should not be used in production outside an isolated, fully trusted environment. Pre-stage reviewed artifacts through the deployment pipeline. Network-isolate the inference server so callers cannot bypass the gateway or reach management endpoints.
Residency lifecycle: registration is not warmth
The diagram is an operational model, not a claim that vLLM exports these exact state names. Distinguish an approved artifact in storage, a CPU-resident adapter and an adapter ready for the next GPU execution. A registered model name proves discoverability, not residency. Instrument the pinned runtime or loader when possible; otherwise label cold/warm state as unknown rather than inferring it from a fast response.
Observe artifact fetch, CPU load, device transfer and queueing separately where instrumentation permits. A CPU-resident adapter may still require work before execution. Conversely, a slow request is not proof of a cache miss: long prefill, scheduling and KV-cache pressure can produce the same symptom. Keep the inference SLO framework and its request-phase measurements intact.
A reproducible workload and configuration manifest
Use the following as a manifest template, not as a ready-to-run configuration or recommended capacity. Replace every REQUIRED field with a recorded value before the test. The example sweep values are experimental choices, not measured safe settings.
experiment_id: adapter-churn-fixture
runtime_image_digest: REQUIRED; gpu_model_and_count: REQUIRED; driver_and_cuda: REQUIRED
base_revision_and_digest: REQUIRED; tokenizer_and_template_digest: REQUIRED
adapter_registry: REQUIRED mapping of immutable alias to digest, base revision, rank, target modules and approval
prompt_corpus_digest: REQUIRED; language_mix: REQUIRED; input_token_buckets: REQUIRED; output_cap: REQUIRED
random_seed: 17; request_count_per_cell: 10000; repetitions: 3
arrival_policy: fixed-rate open-loop; offered_rates: REQUIRED; generator_concurrency_limit: REQUIRED
traffic_profiles: single-adapter baseline, uniform approved set, skewed approved set, popularity reversal
max_loras_sweep: [1, 2, 4]; max_cpu_loras_sweep: [4, 8, 16]; max_lora_rank_sweep: supported ceilings at or above the registry maximum
runtime_dynamic_loading: disabled; cold_start_reset_procedure: REQUIRED
acceptance: per-workload TTFT and failure budgets REQUIRED; identity mismatches allowed: 0
Choose enough approved adapters to test a working set below and above each proposed CPU capacity. If the pinned runtime cannot admit the selected registry with a given configuration, record a configuration rejection, not a latency result. Keep the adapter set fixed when sweeping rank ceilings: changing adapter weights at the same time would confound the comparison. Keep CPU capacity above every max_loras value during the first sweep, then fix max_loras and vary CPU capacity.
Generate identical prompt and output-length distributions across profiles. Use a single-adapter baseline at the same offered rate, a uniform profile across the approved set and a deliberately skewed profile, for example 80% of arrivals to one adapter. Then reverse the popular adapter without changing arrival rate. Record the actual generated request sequence, not only its seed. Open-loop arrivals expose queue growth; report dropped or delayed generator submissions so the load generator cannot hide overload.
Run both warmed and reset trials. Define the reset precisely: restarting the engine is not equivalent to clearing OS page cache or remote artifact caches. For each cell, retain startup outcomes, accepted requests, completions, cancellations, timeouts, failed loads and identity mismatches. Report per-adapter p50/p95/p99 TTFT with sample counts, end-to-end latency, load durations where observable, host/GPU memory and queue depth. A rare adapter may have too few observations for a useful p99 even when the total run contains many requests.
Share-versus-isolate decision table
| Condition | Required evidence | Decision | Cost or limitation |
|---|---|---|---|
| Small stable working set | All workloads pass warm and reset trials | Candidate for shared pool | Shared failure domain remains |
| Long tail exceeds residency capacity | Reload and popularity-reversal trials pass per-adapter SLOs | Share only with measured admission limits | More CPU capacity alone may not fix device churn |
| Rare adapter misses first-token budget | Failures correlate with observed loading or queueing | Warm a dedicated pool or isolate that class | Duplicate base weights and idle capacity |
| Adapter identity mismatch | Expected and executed release disagree | Stop rollout; fix routing before capacity tuning | No latency improvement excuses incorrect identity |
| Different trust or compatibility boundary | Security and base-adapter review require separation | Isolate approved variants | Higher infrastructure and operational cost |
Do not turn the table into automatic promotion based only on percentiles. Require every hard gate, record the workload window and test again after runtime, registry or traffic-policy changes. An acceptable shared pool today can become unsuitable when a new tenant changes the working set.
Routing and cold-load acceptance checklist
1. Unknown and unauthorized adapter: gateway rejects before inference; confirm the backend never receives the request and no base-model fallback occurs.
2. Incompatible rank or base revision: deployment validation rejects it before traffic. Exercise the actual engine load failure in an isolated test and retain the error, not merely a mocked gateway response.
3. Eviction and reload: drive more approved adapters than the tested resident working set, then return to the original adapter. Prove the intended revision executes and record whether loading was actually observed. Do not infer eviction solely from elapsed time.
4. Revision change during streaming: begin a long request on extract-r7, admit new traffic to a distinct extract-r8 alias or replacement pool, and drain r7. The in-flight request must finish on r7; never overwrite mutable weights underneath it. Verify the executed artifact in server-side instrumentation, not only the response's model label.
5. Process restart and missing artifact: start with the documented reset procedure and deliberately remove a required staged artifact in a test environment. Admission must fail closed or route to an independently approved equivalent deployment; no silent substitution.
6. Cancellation and saturation: cancel streams while changing adapter popularity. Verify that request bookkeeping drains, subsequent requests retain correct identity and memory does not grow without bound across repeated trials.
Semantic canaries with distinct expected behaviours can help detect routing mistakes, but output text alone cannot prove adapter identity. Use trusted registry-to-loader correlation and request traces; keep sensitive prompts out of broad metrics and avoid unbounded request-ID labels. A gateway fixture tests authorization and alias resolution, not GPU execution or capacity.
Why Transformers hotswapping is not the same contract
Hugging Face's Transformers PEFT guide describes replacing LoRA weights in an existing adapter slot. For compiled models it describes enable_peft_hotswap before loading and compilation, a target_rank ceiling and possible recompilation when a replacement targets more layers. This is useful for an explicitly coordinated process, but is not evidence that mutating one model object provides vLLM-style per-request isolation.
If using in-place hotswapping, serialize access or build a verified synchronization boundary around the mutation and every generation that uses the slot. Do not copy per-request multi-adapter concurrency assumptions into that design. Equally, do not promise that a vLLM adapter reload preserves every in-flight semantic without testing the pinned implementation. Immutable aliases and drain-and-replace deployments make the intended release boundary easier to verify, at the cost of temporary spare capacity.
Start with a consolidation gate, not a GPU-saving claim
Select one base model and an approved adapter registry, set per-workload latency and error budgets, then run the baseline, uniform, skewed and popularity-reversal trials. Keep a dedicated-pool rollback route. Approve consolidation only when identity checks, cold paths and every workload's service objectives pass; calculate operating cost from that passing configuration, including spare capacity and operational effort.
An adapter-serving architecture review can produce the registry contract, experiment manifest, routing tests and share-versus-isolate decision for your deployment. The useful outcome is not a promised speedup: it is a measured boundary for which variants can safely share capacity, and a clear isolation rule for those that cannot.
