Fuse layout with pointwise op #2886

umangyadav · 2024-03-13T21:40:36Z

No description provided.

migraphx-bot · 2024-03-14T01:29:37Z

Test	Batch	Rate new 769566	Rate old 5032ef	Diff	Compare
torchvision-resnet50	64	3,049.51	3,058.75	-0.30%	✅
torchvision-resnet50_fp16	64	7,111.50	7,127.23	-0.22%	✅
torchvision-densenet121	32	2,449.14	2,451.06	-0.08%	✅
torchvision-densenet121_fp16	32	4,105.29	4,109.97	-0.11%	✅
torchvision-inceptionv3	32	1,649.22	1,659.74	-0.63%	✅
torchvision-inceptionv3_fp16	32	2,617.78	2,619.60	-0.07%	✅
cadene-inceptionv4	16	778.48	780.84	-0.30%	✅
cadene-resnext64x4	16	745.67	745.77	-0.01%	✅
slim-mobilenet	64	6,716.79	6,714.58	0.03%	✅
slim-nasnetalarge	64	175.76	175.97	-0.12%	✅
slim-resnet50v2	64	2,980.90	2,981.85	-0.03%	✅
bert-mrpc-onnx	8	1,071.19	1,070.87	0.03%	✅
bert-mrpc-tf	1	447.45	446.05	0.31%	✅
pytorch-examples-wlang-gru	1	454.56	377.78	20.32%	🔆
pytorch-examples-wlang-lstm	1	352.86	354.57	-0.48%	✅
torchvision-resnet50_1	1	788.51	792.31	-0.48%	✅
cadene-dpn92_1	1	427.72	428.08	-0.08%	✅
cadene-resnext101_1	1	364.05	363.76	0.08%	✅
onnx-taau-downsample	1	348.75	349.31	-0.16%	✅
dlrm-criteoterabyte	1	34.64	34.73	-0.28%	✅
dlrm-criteoterabyte_fp16	1	58.01	57.72	0.50%	✅
agentmodel	1	6,902.12	6,816.03	1.26%	✅
unet_fp16	2	58.33	58.23	0.18%	✅
resnet50v1_fp16	1	990.94	985.92	0.51%	✅
resnet50v1_int8	1	857.69	866.51	-1.02%	✅
bert_base_cased_fp16	64	1,040.83	1,041.04	-0.02%	✅
bert_large_uncased_fp16	32	323.33	321.85	0.46%	✅
bert_large_fp16	1	nan	nan	nan%	❌
distilgpt2_fp16	16	2,027.29	2,026.48	0.04%	✅
yolov5s	1	513.82	514.61	-0.15%	✅
tinyllama	1	44.85	44.75	0.22%	✅
vicuna-fastchat	1	182.79	182.17	0.34%	✅
whisper-tiny-encoder	1	404.57	403.43	0.28%	✅
whisper-tiny-decoder	1	425.17	420.63	1.08%	✅

This build is not recommended to merge 🔴

migraphx-bot · 2024-03-14T01:29:38Z

❌bert-mrpc-onnx: ERROR - check error output

Traceback (most recent call last):
File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 340, in
main()
File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 205, in main
model = migraphx.parse_onnx(model_name, default_dim_value=batch)
RuntimeError: /src/AMDMIGraphX/src/onnx/onnx_parser.cpp:264: parse_from: PARSE_FROM: Failed reading onnx file: /new-saved-models/huggingface-transformers/bert_mrpc1.onnx

✅ bert-mrpc-tf: PASSED: MIGraphX meets tolerance

✅ pytorch-examples-wlang-gru: PASSED: MIGraphX meets tolerance

✅ pytorch-examples-wlang-lstm: PASSED: MIGraphX meets tolerance

✅ torchvision-resnet50_1: PASSED: MIGraphX meets tolerance

✅ cadene-dpn92_1: PASSED: MIGraphX meets tolerance

❌cadene-resnext101_1: ERROR - check error output

2024-03-15 08:05:05.118185744 [W:onnxruntime:, model.cc:183 Model] ONNX Runtime only guarantees support for models stamped with opset version 7 or above for opset domain 'ai.onnx'. Please upgrade your model to opset 7 or higher. For now, this opset 6 model may run depending upon legacy support of some older opset version operators.
2024-03-15 08:05:05.124673770 [W:onnxruntime:, transpose_optimizer.cc:28 ApplyImpl] Transpose optimizer failed: Unsupported ONNX opset: 6
Traceback (most recent call last):
File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 340, in
main()
File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 267, in main
sess = ort.InferenceSession(model_name,
File "/usr/local/lib/python3.8/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 419, in init
self._create_inference_session(providers, provider_options, disabled_optimizers)
File "/usr/local/lib/python3.8/dist-packages/onnxruntime/capi/onnxruntime_inference_collection.py", line 463, in _create_inference_session
sess.initialize_session(providers, provider_options, disabled_optimizers)
onnxruntime.capi.onnxruntime_pybind11_state.NotImplemented: [ONNXRuntimeError] : 9 : NOT_IMPLEMENTED : Could not find an implementation for BatchNormalization(6) node with name ''

✅ dlrm-criteoterabyte: PASSED: MIGraphX meets tolerance

✅ agentmodel: PASSED: MIGraphX meets tolerance

❌unet: ERROR - check error output

Traceback (most recent call last):
File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 340, in
main()
File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 207, in main
model = migraphx.parse_onnx(model_name,
RuntimeError: /src/AMDMIGraphX/src/onnx/onnx_parser.cpp:264: parse_from: PARSE_FROM: Failed reading onnx file: /new-saved-models/unet/model.onnx

✅ resnet50v1: PASSED: MIGraphX meets tolerance

✅ bert_base_cased_fp16: PASSED: MIGraphX meets tolerance

🔴bert_large_uncased_fp16: FAILED: MIGraphX is not within tolerance - check verbose output

❌bert_large: ERROR - check error output

Traceback (most recent call last):
File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 340, in
main()
File "/src/AMDMIGraphX/tools/accuracy/accuracy_checker.py", line 205, in main
model = migraphx.parse_onnx(model_name, default_dim_value=batch)
RuntimeError: /src/AMDMIGraphX/src/onnx/onnx_parser.cpp:264: parse_from: PARSE_FROM: Failed reading onnx file: /new-saved-models/bert/model.onnx

✅ yolov5s: PASSED: MIGraphX meets tolerance

✅ tinyllama: PASSED: MIGraphX meets tolerance

✅ vicuna-fastchat: PASSED: MIGraphX meets tolerance

✅ whisper-tiny-encoder: PASSED: MIGraphX meets tolerance

✅ whisper-tiny-decoder: PASSED: MIGraphX meets tolerance

✅ distilgpt2_fp16: PASSED: MIGraphX meets tolerance

pfultz2 · 2024-03-14T15:02:29Z

src/targets/gpu/fuse_ops.cpp

@@ -767,11 +767,15 @@ struct find_contiguous
    }
 };

-struct find_pointwise_contiguous
+struct find_pointwise_layout


Maybe this should be named find_pointwise_layout_contiguous to make it clear it works for contiguous as well as layout.

pfultz2 · 2024-03-14T15:10:53Z

test/gpu/fuse_ops.cpp

+    migraphx::shape s1{migraphx::shape::float_type, {2, 320, 128, 128}};
+    migraphx::shape s2{migraphx::shape::float_type, {320, 320, 3, 3}, {2880, 1, 960, 320}};
+    migraphx::shape s3{migraphx::shape::float_type, {2, 320, 128, 128}, {5242880, 1, 40960, 320}};
+    migraphx::shape s4{migraphx::shape::int8_type, {41943040}}; //  workspace for gpu::convolution


I dont think its a good idea to hardcode the workspace which could change. Maybe its better to run lowering(and possibly fuse_pointwise).

This is just test for the pass. It will not invoke "compute()". Therefore i think it is okay to have any value for workspace. I can put a comment about that.

codecov · 2024-03-14T20:01:50Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.84%. Comparing base (5032ef3) to head (7695661).
Report is 1 commits behind head on develop.

Additional details and impacted files

@@           Coverage Diff            @@
##           develop    #2886   +/-   ##
========================================
  Coverage    91.84%   91.84%           
========================================
  Files          478      478           
  Lines        18179    18179           
========================================
  Hits         16696    16696           
  Misses        1483     1483

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

add tests

69c1554

umangyadav requested a review from causten as a code owner March 13, 2024 21:40

umangyadav requested review from pfultz2, kahmed10 and shivadbhavsar and removed request for causten March 13, 2024 21:40

umangyadav self-assigned this Mar 13, 2024

umangyadav added 2 commits March 13, 2024 21:41

remove find_pointwise_contiguous

e590405

add verify test

d16aafa

umangyadav added the Perf Improve label Mar 13, 2024

Merge branch 'develop' into fuse_layout

7802ec3

pfultz2 reviewed Mar 14, 2024

View reviewed changes

rename, add comments

38696e7

umangyadav requested a review from pfultz2 March 14, 2024 19:54

fix tidy

7676bd8

kahmed10 approved these changes Mar 14, 2024

View reviewed changes

pfultz2 approved these changes Mar 14, 2024

View reviewed changes

Merge branch 'develop' into fuse_layout

7695661

causten merged commit 7460ac3 into develop Mar 15, 2024
48 checks passed

causten deleted the fuse_layout branch March 15, 2024 14:13

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fuse layout with pointwise op #2886

Fuse layout with pointwise op #2886

umangyadav commented Mar 13, 2024

migraphx-bot commented Mar 14, 2024 •

edited

Loading

migraphx-bot commented Mar 14, 2024 •

edited

Loading

pfultz2 Mar 14, 2024

umangyadav Mar 14, 2024

pfultz2 Mar 14, 2024

umangyadav Mar 14, 2024 •

edited

Loading

codecov bot commented Mar 14, 2024 •

edited

Loading

Fuse layout with pointwise op #2886

Fuse layout with pointwise op #2886

Conversation

umangyadav commented Mar 13, 2024

migraphx-bot commented Mar 14, 2024 • edited Loading

migraphx-bot commented Mar 14, 2024 • edited Loading

pfultz2 Mar 14, 2024

Choose a reason for hiding this comment

umangyadav Mar 14, 2024

Choose a reason for hiding this comment

pfultz2 Mar 14, 2024

Choose a reason for hiding this comment

umangyadav Mar 14, 2024 • edited Loading

Choose a reason for hiding this comment

codecov bot commented Mar 14, 2024 • edited Loading

Codecov Report

migraphx-bot commented Mar 14, 2024 •

edited

Loading

migraphx-bot commented Mar 14, 2024 •

edited

Loading

umangyadav Mar 14, 2024 •

edited

Loading

codecov bot commented Mar 14, 2024 •

edited

Loading