Skip to content

add runtime symbol resolution op#5085

Open
shivadbhavsar wants to merge 9 commits into
developfrom
sym_resolve_op
Open

add runtime symbol resolution op#5085
shivadbhavsar wants to merge 9 commits into
developfrom
sym_resolve_op

Conversation

@shivadbhavsar

@shivadbhavsar shivadbhavsar commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Motivation

For dynamic shapes, the runtime needs to be able to evaluate symbols using concrete input dimensions to be able to compute dynamic ops (eg. slice)

Technical Details

Attributes (std::vector<sym::expr>, reflected directly):

  • expressions — symbolic expressions to evaluate.
    Inputs: one tensor whose original compile-time shape contains the referenced root symbols as direct dimensions.
    At runtime, each direct symbolic dimension is bound to the corresponding concrete input lens:
compile-time: {N, 3, H, W}
runtime:      {7, 3, 10, 12}
bindings:     N=7, H=10, W=12

Output: one packed int64 vector of shape {expressions.size()}. Element i is eval(expressions[i]), unclamped.
Implementation:

  • compute_shape requires one input, verifies every referenced root symbol appears as a direct input dimension, and returns int64_type, {expressions.size()}.
  • compute reads the original input shape from dyn_output::input_shapes and the concrete dimensions from the runtime argument shape.
  • It maps each direct root symbol to its runtime lens, verifies repeated symbols resolve consistently, then evaluates every expression with eval_uint.
    Restriction: compound input dimensions are not inverted. For example, an input dimension N/2 cannot be used to recover N; N must appear directly on an input axis.
    Example: expressions=[N, H/2, W/2], compile-time input {N,3,H,W}, runtime input {7,3,10,12}[7,5,6].

Usage

x = @param:x -> float_type, {N[1..16], 3, H[1..32], W[1..32]}, {3*H*W, H*W, W, 1}
@1 = eval_expr[expressions={N, H/2, W/2}](x) -> int64_type, {3}, {1}
@2 = slice[axes={0, 2, 3},starts={0, 0, 0}](x,@1) -> dynamic, float_type, {[0..16], 3, [0..32], [0..32]}
@3 = @return(@2)

Changelog Category

Add a CHANGELOG.md entry for any option other than Not Applicable

    • Added: New functionality.
    • Changed: Changes to existing functionality.
    • Removed: Functionality or support that has been removed. (Compared to a previous release)
    • Optimized: Component performance that has been optimized or improved.
    • Resolved Issues: Known issues from a previous version that have been resolved.
    • Not Applicable: This PR is not to be included in the changelog.

Follow the LLVM AI Tool Use Policy for contributions using AI.

@shivadbhavsar
shivadbhavsar requested a review from CharlieL7 July 21, 2026 22:15
@shivadbhavsar shivadbhavsar self-assigned this Jul 21, 2026
@shivadbhavsar
shivadbhavsar requested a review from causten as a code owner July 21, 2026 22:15
Copilot AI review requested due to automatic review settings July 21, 2026 22:15

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a new resolve_sym_expr runtime operator to evaluate symbolic dimension expressions from concrete runtime dimension values, enabling dynamic-shape lowering patterns (e.g., feeding symbolic slice bounds as runtime inputs).

Changes:

  • Added op::resolve_sym_expr (reflected attributes exprs/symbols) that returns a tuple of evaluated int64 scalars.
  • Added reference and shape tests for resolve_sym_expr, plus an integration-style ref test demonstrating use with dynamic multi-input slice.
  • Registered the new operator in the build system.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/include/migraphx/op/resolve_sym_expr.hpp New operator implementation for runtime symbol expression evaluation.
src/CMakeLists.txt Registers resolve_sym_expr in register_migraphx_ops().
test/ref/resolve_sym_expr.cpp New ref tests covering single/multi-symbol evaluation behavior.
test/op_shape_test.cpp New shape tests (valid shape + bad input arity) for the operator.
test/ref/slice.cpp New ref test demonstrating resolve_sym_expr feeding a runtime slice bound.

Comment thread src/include/migraphx/op/resolve_sym_expr.hpp Outdated
Comment thread src/include/migraphx/op/resolve_sym_expr.hpp Outdated
Comment thread test/ref/resolve_sym_expr.cpp Outdated
@gh-app-migraphx-bot-pr-write

gh-app-migraphx-bot-pr-write Bot commented Jul 22, 2026

Copy link
Copy Markdown
Test Batch New Rate (4c03db) Old Rate (5c50d0)* Diff Status
torchvision-resnet50 64 3,264.14 3,278.37 -0.43%
torchvision-resnet50_fp16 64 7,544.73 7,611.38 -0.88%
torchvision-densenet121 32 2,482.20 2,501.41 -0.77%
torchvision-densenet121_fp16 32 4,997.39 5,004.64 -0.14%
torchvision-inceptionv3 32 2,058.30 2,059.27 -0.05%
torchvision-inceptionv3_fp16 32 4,415.10 4,417.65 -0.06%
cadene-inceptionv4 16 821.29 819.80 0.18%
cadene-resnext64x4 16 782.34 783.52 -0.15%
slim-mobilenet 64 8,384.84 8,394.63 -0.12%
slim-nasnetalarge 64 228.86 197.60 15.82% 🔆
slim-resnet50v2 64 3,181.56 3,185.87 -0.14%
bert-mrpc-onnx 8 1,168.15 1,161.65 0.56%
bert-mrpc-tf 1 488.68 486.30 0.49%
pytorch-examples-wlang-gru 1 497.24 469.65 5.87% 🔆
pytorch-examples-wlang-lstm 1 405.68 379.72 6.84% 🔆
torchvision-resnet50_1 1 1,027.20 1,023.59 0.35%
cadene-dpn92_1 1 445.07 436.69 1.92%
cadene-resnext101_1 1 363.01 364.25 -0.34%
onnx-taau-downsample 1 846.67 845.97 0.08%
dlrm-criteoterabyte 1 32.38 32.54 -0.50%
dlrm-criteoterabyte_fp16 1 51.80 52.59 -1.50%
agentmodel 1 8,993.37 9,452.06 -4.85%
unet_fp16 2 58.81 59.01 -0.34%
resnet50v1_fp16 1 1,381.64 1,378.39 0.24%
resnet50v1_int8 1 1,744.74 1,626.43 7.27% 🔆
bert_base_cased_fp16 64 1,097.54 1,102.97 -0.49%
bert_large_uncased_fp16 32 345.70 346.63 -0.27%
bert_large_fp16 1 202.31 205.51 -1.56%
distilgpt2_fp16 16 2,091.39 2,098.82 -0.35%
yolov5s 1 560.05 562.00 -0.35%
tinyllama 1 45.79 45.93 -0.29%
vicuna-fastchat 1 44.01 44.06 -0.11%
whisper-tiny-encoder 1 412.61 415.56 -0.71%
whisper-tiny-decoder 1 414.86 410.79 0.99%
llama2_7b 1 20.83 21.01 -0.85%
qwen1.5-7b 1 23.56 23.60 -0.19%
phi3-3.8b 1 26.71 26.80 -0.36%
llama3-8b 1 21.75 21.70 0.22%
whisper-large-encoder 1 10.18 10.22 -0.41%
whisper-large-decoder 1 106.38 104.52 1.78%
mistral-7b 1 23.74 23.74 -0.01%
FLUX.1-schnell 1 765.69 723.54 5.83% 🔆

Check flagged results 🔆

* No develop baseline was found for this PR's branch point; compared against the latest available develop run instead.

@gh-app-migraphx-bot-pr-write

gh-app-migraphx-bot-pr-write Bot commented Jul 22, 2026

Copy link
Copy Markdown
Test Status Result
bert-mrpc-onnx PASSED: MIGraphX meets tolerance
bert-mrpc-tf PASSED: MIGraphX meets tolerance
pytorch-examples-wlang-gru PASSED: MIGraphX meets tolerance
pytorch-examples-wlang-lstm PASSED: MIGraphX meets tolerance
dlrm-criteoterabyte PASSED: MIGraphX meets tolerance
agentmodel PASSED: MIGraphX meets tolerance
unet PASSED: MIGraphX meets tolerance
resnet50v1 PASSED: MIGraphX meets tolerance
bert_base_cased_fp16 PASSED: MIGraphX meets tolerance
bert_large_uncased_fp16 🔴 FAILED: MIGraphX is not within tolerance - check verbose output
bert_large PASSED: MIGraphX meets tolerance
yolov5s PASSED: MIGraphX meets tolerance
tinyllama PASSED: MIGraphX meets tolerance
vicuna-fastchat PASSED: MIGraphX meets tolerance
whisper-tiny-encoder PASSED: MIGraphX meets tolerance
whisper-tiny-decoder PASSED: MIGraphX meets tolerance
llama2_7b PASSED: MIGraphX meets tolerance
qwen1.5-7b PASSED: MIGraphX meets tolerance
phi3-3.8b PASSED: MIGraphX meets tolerance
llama3-8b PASSED: MIGraphX meets tolerance
whisper-large-encoder PASSED: MIGraphX meets tolerance
whisper-large-decoder PASSED: MIGraphX meets tolerance
mistral-7b PASSED: MIGraphX meets tolerance
FLUX.1-schnell PASSED: MIGraphX meets tolerance

Comment thread src/include/migraphx/op/resolve_sym_expr.hpp Outdated
@shivadbhavsar
shivadbhavsar requested a review from pfultz2 July 23, 2026 22:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants