Edge Model Conversion: Prove Numerical Parity Before Releasing ONNX

BLUF: A valid ONNX file is not evidence that an edge model still makes the same decision. Release conversion, quantization and target-provider execution as separate changes. Compare frozen input/output tensors, reject non-finite or shape-mismatched results, and test accept/review/reject crossings independently of numerical tolerance. A tiny score deviation can pass a tensor check while changing a part disposition.

This is an implementation pattern for a bounded industrial advisory score, not a claim of factory deployment. The executable example below tests the comparison gate with synthetic outputs only. It does not export or execute an ONNX model, measure field accuracy, or validate a GPU/NPU. The production workflow explicitly requires those additional runs on your actual release artifacts and hardware.

One export contains several different release questions

The ONNX checker documentation describes checks on model consistency; full_check also runs shape inference. Neither step evaluates your held-out production cases. A legal graph can still consume channels in the wrong order, expose a changed output shape or shift a score across a disposition boundary.

Keep three comparisons separate. First compare the frozen source-framework model against its floating-point ONNX export with identical prepared inputs. Next compare the approved floating-point ONNX graph with a quantized derivative under a declared calibration and optimization configuration. Finally run the exact approved artifact using the intended target execution provider and compare it with the approved reference. If you combine all changes, a failure becomes difficult to localize.

The ONNX Runtime execution-provider documentation explains that providers claim supported nodes or subgraphs through GetCapability. An ordered CUDA/CPU provider list can execute unsupported CUDA work on CPU. A registered provider is therefore not proof that every node ran on that provider. Record actual placement where profiling or runtime logs expose it, plus any blind spots. Unapproved fallback is a hold, not a performance footnote.

Freeze the tensor contract before comparing scores

For a one-output advisory classifier, define inputs as float32 batches with a fixed feature order and an explicit allowed batch-size range. For example, vibration RMS in mm/s and bearing temperature in degrees Celsius are distinct channels; a unit conversion or reordered channel is an input-contract change, not numerical noise. Freeze scaling coefficients, clipping, preprocessing code, missing-value policy and feature acquisition window. Reject invalid inputs before inference unless a separately approved imputation rule applies.

Preserve both raw test records and the prepared golden tensors. The raw-record replay tests preprocessing; the tensor replay isolates executable-model behavior. Hash each manifest and keep stable sample IDs, expected dtype, dimensions, output meaning and reference-runtime identity. Do not silently flatten a tensor before validating its axes: equal element counts can hide a transposed output. Our small comparator below deliberately accepts only flat, one-score-per-sample lists; adapters must verify richer tensor structure before calling it.

The existing industrial data-contract guide addresses semantic meaning upstream. Here the release boundary is narrower: can the new executable preserve the output contract and the approved downstream decision for the exact same input?

A staged parity pipeline

At each stage, retain the input digest, both output tensors, comparison policy and failing sample IDs. Do not replace the reference with the most recent candidate merely because it loads successfully. Optimization settings are part of the candidate: operator fusion and different kernels can change floating-point accumulation. Test the intended production optimization configuration, not only a debugging configuration with optimizations disabled.

Boundary fixtures should cover values on either side of every downstream threshold, exact ties, all-zero valid inputs, constant channels, maximum permitted batch size, batch size one and every approved dynamic-shape family. Degenerate does not mean valid: empty batches, NaN, infinity and unsupported shapes should exercise the rejection path. Invalid data must not be counted as a numerical pass because a comparison library happens to tolerate NaNs.

Decision table: conversion is not quantization

For the quantized branch, use a distinct tolerance policy; do not loosen the conversion tolerance to hide quantization loss. A new quantization recipe may legitimately change scores, but accepting those changes requires task-level evidence and an approved decision policy. This article chooses a conservative zero-unreviewed-crossing release gate, not a universal mathematical requirement that all quantized outputs be identical.

An executable comparison gate with a deliberate trap

Use the elementwise rule abs(candidate − reference) ≤ atol + rtol × abs(reference). The reference defines the scale. Near zero, the absolute term matters; a pure relative error can become meaningless. Declare units and tolerances separately for each output. The illustrative values below, atol = 0.000001 and rtol = 0.00001, are teaching choices, not recommended limits for your plant.

For an illustrative score, accept below 0.2, review from 0.2 through 0.8, and reject above 0.8. These thresholds are not calibrated defect probabilities. They exist to demonstrate a release failure. The inspection-calibration article covers how to justify real thresholds; parity testing must preserve the already-approved policy rather than re-optimize it on conversion failures.

The following standard-library Python snippet is executable in order. It checks equal nonempty lengths, finite values, numerical tolerance and downstream decisions. It handles the deliberately bounded flat-output contract only. Save the code lines together and run them with python3.

import math

atol, rtol = 1e-6, 1e-5

action = lambda x: 'accept' if x < .2 else 'reject' if x > .8 else 'review'

ref, got = [.1999999, .8000001], [.2000001, .7999999]

valid = bool(ref) and len(ref) == len(got)

valid = valid and all(math.isfinite(x) for x in ref + got)

close = valid and all(abs(a-b) <= atol + rtol*abs(a) for a,b in zip(ref,got))

flips = sum(action(a) != action(b) for a,b in zip(ref,got)) if valid else None

release = close and flips == 0

assert close and flips == 2 and not release

print({'numeric_pass': close, 'decision_flips': flips, 'release': release})

Executed result: numeric_pass = True, decision_flips = 2, release = False. Each deviation is approximately 0.0000002, yet both samples change disposition. One moves from accept to review; the other moves from reject to review. Reporting only mean absolute error would conceal the reason to hold this candidate.

The accompanying editorial self-test ran eleven synthetic cases: ordinary values, near-zero values, threshold crossings, a larger deviation, length mismatch, empty input, NaN, infinity, one-element output, three-element output and exact boundary ties. All eleven produced their expected pass/hold outcomes. This validates the comparator behavior on those cases only. The different list lengths are not evidence that an ONNX graph supports dynamic shapes. No ONNX Runtime, source-framework export or quantized model was executed for these results.

Turn the self-test into artifact evidence

Collect source outputs in evaluation/inference mode from the frozen model and prepared tensors. Record random seeds and any remaining nondeterministic operations; setting a seed alone does not prove deterministic execution. Export with explicit input/output names, supported opset and shape constraints. Run the structural checker, then collect outputs from the exported graph on the reference provider. Join by immutable sample ID, validate shapes and dtypes, and run the comparison gate per named output.

Repeat for the quantized artifact, then on the target device. Reuse identical held-out acceptance inputs; do not tune quantization on the final acceptance failures and call that set independent. Static quantization uses calibration data to determine activation ranges. Keep its provenance distinct from acceptance data and include representative operating regimes. Quantization is not lossless, and the official quantization guide explicitly warns about accuracy loss and hardware-dependent performance. A smaller artifact can still run slower because of conversion overhead or unsupported kernels.

When divergence appears, isolate the first failing stage. For supported float/quantized graph pairs, ONNX Runtime exposes weight and activation matching through qdq_loss_debug, including create_weight_matching and create_activation_matching. These are debugging aids, not a promise that arbitrary source-framework and exported tensors can be paired. Graph transformations can make correspondence difficult. Preserve a preprocessed float graph and separate optimization from quantization where practical so the comparison remains interpretable.

Do not declare the target validated from a CPU run. Capture the provider build, driver/firmware, device identity, provider options and node placement where available. Measure warm and cold behavior, latency distribution, memory and transfer overhead under the relevant input shapes. Numerical agreement does not establish a deadline guarantee; a CPU fallback can preserve outputs while violating operational timing.

Edge release manifest and acceptance checklist

Identity: source revision, model and external-weight digests, exported/quantized artifact digests, opset, ONNX IR version, converter version and complete runtime dependency lock.

Contract: input/output names, dtypes, axis meanings, approved shapes, units, preprocessing digest, missing-value handling, golden-input manifest and reference-output digest.

Execution: hardware identity, provider/runtime versions, provider priority and options, optimization level, driver/firmware, observed node placement and explicitly unobservable placement details.

Acceptance: per-output absolute/relative tolerances, threshold and tie policy, decision-flip report, invalid-input results, held-out task metrics, latency/memory results, exclusions and reviewer decision.

Operation: approved scope, owner, rollback artifact and compatibility proof. Bind this evidence into the model nameplate and rollback bundle, rather than storing an untraceable spreadsheet beside a mutable model tag.

Release only when every required comparison has an explicit result and every deviation has an accountable disposition. Keep advisory inference outside the independent safety-control path. Numerical parity cannot establish safety integrity, conformity, field accuracy or acceptable defect-escape risk. A biased source model can be reproduced perfectly.

Start with one conversion boundary

Choose one model, one frozen preprocessing contract and one target device. Build the golden-input pack, inject a threshold-crossing failure, then run the real source/export/target comparisons before broadening the hardware fleet. If you need a technical review, bring the artifacts, provider configuration and current acceptance policy: I can help structure a conversion gate and release manifest that your engineering and operations teams can reproduce. A useful first outcome is a precise hold reason, not a green check obtained by widening every tolerance.