Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump onnxruntime from 1.10.0 to 1.16.1 in /tools/accuracy #2321

Merged
merged 3 commits into from
Oct 19, 2023

Conversation

dependabot[bot]
Copy link
Contributor

@dependabot dependabot bot commented on behalf of github Oct 12, 2023

Bumps onnxruntime from 1.10.0 to 1.16.1.

Release notes

Sourced from onnxruntime's releases.

ONNX Runtime v1.16.1

This release fixed some issues in 1.16.0

ONNX Runtime v1.16.0

General

  • Support for serialization of models >=2GB

APIs

  • New session option to disable default CPU EP fallback session.disable_cpu_ep_fallback
  • Java
    • Support for fp16 and bf16 tensors as inputs and outputs, along with utilities to convert between these and fp32 data. On JDK 20 and newer the fp16 conversion methods use the JDK's Float.float16ToFloat and Float.floatToFloat16 methods which can be hardware accelerated and vectorized on some platforms.
    • Support for external initializers so that large models that can be instantiated without filesystem access
  • C#
    • Expose OrtValue API as the new preferred API to run inference in C#. This reduces garbage and exposes direct native memory access via Slice like interfaces.
    • Make Float16 and BFloat16 full featured fp16 interfaces that support conversion and expose floating properties (e.g. IsNaN, IsInfinity, etc)
  • C++
    • Make Float16_t and BFloat16_t full featured fp16 interfaces that support conversion and expose floating properties (e.g. IsNaN, IsInfinity, etc)

Performance

  • Improve LLM quantization accuracy with smoothquant
  • Support 4-bit quantization on CPU
  • Optimize BeamScore to improve BeamSearch performance
  • Add FlashAttention v2 support for Attention, MultiHeadAttention and PackedMultiHeadAttention ops

Execution Providers

  • CUDA EP
    • Initial fp8 support (QDQ, Cast, MatMul)
    • Relax CUDA Graph constraints to allow more models to utilize
    • Allow CUDA allocator to be registered with ONNX Runtime externally
  • TensorRT EP
    • CUDA Graph support
    • Support user provided cuda compute stream
    • Misc bug fixes and improvements
  • OpenVINO EP
    • Support OpenVINO 2023.1
  • QNN EP
    • Enable context binary cache to reduce initialization time
    • Support QNN 2.12
    • Support for resize with asymmetric transformation mode on HTP backend
    • Ops support: Equal, Less, LessOrEqual, Greater, GreaterOrEqual, LayerNorm, Asin, Sign, DepthToSpace, SpaceToDepth
    • Support 1D Conv/ConvTranspose
    • Misc bug fixes and improvements

Mobile

  • Initial support for Azure EP
  • Dynamic shape support for CoreML
  • Improve React Native performance with JSI
  • Mobile support for CLIPImageProcessor pre-processing and CLIP scenario
  • Swift Package Manager support for ONNX Runtime inference and ONNX Runtime extensions via onnxruntime-swift-package-manager

... (truncated)

Commits

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

@dependabot dependabot bot added the dependencies Pull requests that update a dependency file label Oct 12, 2023
@codecov
Copy link

codecov bot commented Oct 12, 2023

Codecov Report

Merging #2321 (1e41a78) into develop (6e86734) will not change coverage.
Report is 5 commits behind head on develop.
The diff coverage is n/a.

❗ Current head 1e41a78 differs from pull request most recent head 88672d3. Consider uploading reports for the commit 88672d3 to get more accurate results

@@           Coverage Diff            @@
##           develop    #2321   +/-   ##
========================================
  Coverage    91.36%   91.36%           
========================================
  Files          439      439           
  Lines        16493    16493           
========================================
  Hits         15069    15069           
  Misses        1424     1424           

@dependabot dependabot bot force-pushed the dependabot/pip/tools/accuracy/onnxruntime-1.16.1 branch 4 times, most recently from 960320a to d4bed64 Compare October 14, 2023 01:17
Bumps [onnxruntime](https://github.com/microsoft/onnxruntime) from 1.10.0 to 1.16.1.
- [Release notes](https://github.com/microsoft/onnxruntime/releases)
- [Changelog](https://github.com/microsoft/onnxruntime/blob/main/docs/ReleaseManagement.md)
- [Commits](microsoft/onnxruntime@v1.10.0...v1.16.1)

---
updated-dependencies:
- dependency-name: onnxruntime
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
@dependabot dependabot bot force-pushed the dependabot/pip/tools/accuracy/onnxruntime-1.16.1 branch from d4bed64 to 5ffec85 Compare October 17, 2023 13:04
@migraphx-bot
Copy link
Collaborator

Test Batch Rate new
44ea53
Rate old
e8e142
Diff Compare
torchvision-resnet50 64 2,850.82 2,850.87 -0.00%
torchvision-resnet50_fp16 64 6,478.75 6,483.76 -0.08%
torchvision-densenet121 32 2,101.06 2,107.54 -0.31%
torchvision-densenet121_fp16 32 3,674.12 3,673.03 0.03%
torchvision-inceptionv3 32 1,594.29 1,594.82 -0.03%
torchvision-inceptionv3_fp16 32 2,593.49 2,592.86 0.02%
cadene-inceptionv4 16 707.26 707.31 -0.01%
cadene-resnext64x4 16 697.56 697.36 0.03%
slim-mobilenet 64 8,352.49 8,344.55 0.10%
slim-nasnetalarge 64 226.95 226.95 0.00%
slim-resnet50v2 64 2,676.45 2,676.49 -0.00%
bert-mrpc-onnx 8 824.77 824.50 0.03%
bert-mrpc-tf 1 388.88 388.61 0.07%
pytorch-examples-wlang-gru 1 297.68 299.61 -0.64%
pytorch-examples-wlang-lstm 1 313.27 316.43 -1.00%
torchvision-resnet50_1 1 599.60 597.52 0.35%
torchvision-inceptionv3_1 1 337.59 340.06 -0.73%
cadene-dpn92_1 1 394.90 399.86 -1.24%
cadene-resnext101_1 1 330.39 329.54 0.26%
slim-vgg16_1 1 464.17 463.53 0.14%
slim-mobilenet_1 1 2,043.94 2,046.07 -0.10%
slim-inceptionv4_1 1 216.11 214.62 0.70%
onnx-taau-downsample 1 305.88 307.04 -0.38%
dlrm-criteoterabyte 1 21.71 21.71 -0.03%
dlrm-criteoterabyte_fp16 1 40.65 40.69 -0.10%
agentmodel 1 5,828.05 5,783.52 0.77%
unet_fp16 2 55.95 55.96 -0.01%
resnet50v1_fp16 1 953.20 948.45 0.50%
bert_base_cased_fp16 64 970.16 970.32 -0.02%
bert_large_uncased_fp16 32 304.82 304.83 -0.00%
bert_large_fp16 1 167.00 166.60 0.24%
distilgpt2_fp16 16 1,278.21 1,279.20 -0.08%

This build is OK for merge ✅

@migraphx-bot
Copy link
Collaborator


    :white_check_mark:bert-mrpc-onnx: PASSED: MIGraphX meets tolerance

    :white_check_mark:bert-mrpc-tf: PASSED: MIGraphX meets tolerance

    :white_check_mark:pytorch-examples-wlang-gru: PASSED: MIGraphX meets tolerance

    :white_check_mark:pytorch-examples-wlang-lstm: PASSED: MIGraphX meets tolerance

    :white_check_mark:torchvision-resnet50_1: PASSED: MIGraphX meets tolerance

🔴torchvision-inceptionv3_1: FAILED: MIGraphX is not within tolerance - check verbose output


    :white_check_mark:cadene-dpn92_1: PASSED: MIGraphX meets tolerance

    :white_check_mark:cadene-resnext101_1: PASSED: MIGraphX meets tolerance

    :white_check_mark:slim-vgg16_1: PASSED: MIGraphX meets tolerance

    :white_check_mark:slim-mobilenet_1: PASSED: MIGraphX meets tolerance

🔴slim-inceptionv4_1: FAILED: MIGraphX is not within tolerance - check verbose output


    :white_check_mark:dlrm-criteoterabyte: PASSED: MIGraphX meets tolerance

    :white_check_mark:agentmodel: PASSED: MIGraphX meets tolerance

    :white_check_mark:unet: PASSED: MIGraphX meets tolerance

    :white_check_mark:resnet50v1: PASSED: MIGraphX meets tolerance

🔴bert_base_cased_fp16: FAILED: MIGraphX is not within tolerance - check verbose output


🔴bert_large_uncased_fp16: FAILED: MIGraphX is not within tolerance - check verbose output


    :white_check_mark:bert_large: PASSED: MIGraphX meets tolerance

🔴distilgpt2_fp16: FAILED: MIGraphX is not within tolerance - check verbose output

@causten causten merged commit 3d10161 into develop Oct 19, 2023
10 of 11 checks passed
@causten causten deleted the dependabot/pip/tools/accuracy/onnxruntime-1.16.1 branch October 19, 2023 14:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dependencies Pull requests that update a dependency file
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants