Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check onnx models more easily #2378

Merged

Conversation

AlexandreEichenberger
Copy link
Collaborator

Added a new python script built on top of runONNXModel.py to quickly compare the results of a single model compiled under 2 set of different options.

Added .md info as well as a small example in the header file, copied below

# This script is to run and debug an onnx model. Model is run twice, once with
# safe compile options, and the second time with the options to tests. The
# script verifies that both output are the same.

# This script can be used as follows:
#
# CheckONNXModel.py --model=reducegpt2.mlir --test-compile-args="-O3 -march=x86-64" --shape-info=0:10x20
#
# It will compile and run the model reducegpt2.mlir twice.
# * Once with the default (-O0) option, which can be overridden with
#   --ref-compile-args. This will build reference results stored by default
#   in a subdir named "check-ref", which can be overridden with --save-ref="name"
#   option. Building the reference values can be skipped with "--skip-ref"
#   option if it was previously built with the same compile options.
# * Once with the default (-O3) option, which can be overridden with
#   --test-compile-args. The values of this run are compared with the reference
#   values.
#
# Script will fail if the values are not identical. Currently only the
# "--verify-every-value" option is supported.
#
# This script relies on RunONNXModel.py to compile, run, and test. The actual
# commands used by this script are printed on stdout, so that users may call
# them manually if they wish to employ more RunONNXModel.py options.
# 

Signed-off-by: Alexandre Eichenberger <[email protected]>
Copy link
Collaborator

@tungld tungld left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Thanks for adding another useful tool!

@AlexandreEichenberger AlexandreEichenberger merged commit 7b92fbc into onnx:main Jul 19, 2023
4 checks passed
@jenkins-droid
Copy link
Collaborator

Jenkins Linux ppc64le Build #11008 [push] Check onnx models more e... started at 18:43

@jenkins-droid
Copy link
Collaborator

Jenkins Linux s390x Build #12014 [push] Check onnx models more e... started at 18:37

@jenkins-droid
Copy link
Collaborator

Jenkins Linux amd64 Build #12001 [push] Check onnx models more e... started at 17:37

@jenkins-droid
Copy link
Collaborator

Jenkins Linux amd64 Build #12001 [push] Check onnx models more e... passed after 1 hr 15 min

@jenkins-droid
Copy link
Collaborator

Jenkins Linux s390x Build #12014 [push] Check onnx models more e... passed after 1 hr 27 min

@jenkins-droid
Copy link
Collaborator

Jenkins Linux ppc64le Build #11008 [push] Check onnx models more e... passed after 1 hr 50 min

negiyas pushed a commit to negiyas/onnx-mlir that referenced this pull request Jul 31, 2023
* check onnx models more easily

Signed-off-by: Alexandre Eichenberger <[email protected]>
Signed-off-by: Yasushi Negishi <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants