Qiming Bao · Blog · 12 November 2026 · English / 中文

Three ONNX graph details that tools get wrong

This week I fixed three bugs in ONNX tooling: two in ONNX Runtime's graph optimizer and one in my own exporter code. None of them crashed anything. A model ran a little slower than it should, or a test failed only in CI. All three come from small details of the ONNX format that are easy to miss when you write a graph pass or an exporter.

1. An empty optional input is not the same as an omitted one

ONNX lets you skip an optional input in two ways. You can leave it off the end of the input list, Conv(x, w), or you can write it as an empty name, Conv(x, w, ""). Both mean "no bias", and both are valid.

ONNX Runtime's Conv+BatchNormalization, Conv+Add and Conv+Mul fusions checked for a bias with InputDefs().size() == 3. With "" the Conv has three input defs, so the fusion took the "has bias" path. It then asked whether the bias was a constant, got "no" for the empty input, and skipped the fusion at every optimization level (microsoft/onnxruntime#34139). Treating it as "no bias" isn't enough either. The no-bias path appended the new bias with a helper that only adds an input at the end of the list, and slot 2 was already taken by "".

The fix (microsoft/onnxruntime#34141) checks InputDefs()[2]->Exists() as well and sets the new bias with a helper that fills an empty slot or an omitted one.

Rule for graph passes: an optional input is present only if the slot exists and has a non-empty name. Counting inputs is not enough.

2. A sliced Shape only needs the dims it selects

Since opset 15, Shape takes start and end, so Shape(x, start=1) returns every dim except the first. Exported models do this a lot, usually to keep a symbolic batch dim out of a Reshape.

ONNX Runtime's constant folding collected the values of all dims first and gave up at the first symbolic one, before it applied the slice. For x: [N, 3], Shape(x, start=1, end=2) is always [3], but it was not folded because of N (microsoft/onnxruntime#34140). The fix (microsoft/onnxruntime#34142) clamps start/end against the rank first and only requires the selected dims to be known:

Rule for graph passes: apply an op's own attributes before you decide that its input is "not static enough".

3. onnx before 1.21 rejects Constant(value_ints) as Pad's pads

This one was my own bug. I wrote a torchlib implementation of adaptive_avg_pool for onnxscript (microsoft/onnxscript#3201), and it uses a Pad whose pads came from op.Constant(value_ints=[...]). All tests passed locally. In CI, almost every job failed with:

[ShapeInferenceError] 'pads' input must be a 1D (shape: [2 * num_axes]) tensor of type int64

CI pins onnx==1.18, and I had onnx 1.23 locally. A minimal model with only a Pad shows the difference. The pads are the same six int64 values, built in two ways:

Passing the pads as a tensor works on every version. More generally, run your tests with the dependency versions that CI pins, not only the ones in your local environment. I now keep a virtual environment that matches the onnxscript CI pins and run new changes there before I push.

A pattern

All three bugs hide behind a working default case. Most models omit an unused bias instead of writing "", most Shape nodes take the whole shape, and most developers have a recent onnx. Tests that only build the common form pass. For each of these fixes, the useful test was the one that built the uncommon but valid form on purpose and compared it with the common one.

I used Claude (Anthropic) to help write the fixes, build the reproductions and draft this post. I reviewed the results and ran the tests myself.

Qiming Bao is an AI researcher in Auckland, New Zealand, working on LLM reasoning and model deployment. He received his PhD from the University of Auckland's Strong AI Lab and contributes to the PyTorch ONNX exporter, ONNX Runtime, ONNX Runtime GenAI and Microsoft PyRIT.