Add flag to accept non-uniform WG sizes #2167

umangyadav · 2023-09-11T21:24:53Z

fixes #1986
Depends on #2168
Relevant issues:
ROCm/MIOpen#2307

src/targets/gpu/CMakeLists.txt

migraphx-bot · 2023-09-11T23:59:19Z

Test	Batch	Rate new 188cf9	Rate old e8e142	Diff	Compare
torchvision-resnet50	64	2,849.69	2,850.87	-0.04%	✅
torchvision-resnet50_fp16	64	6,475.69	6,483.76	-0.12%	✅
torchvision-densenet121	32	2,101.46	2,107.54	-0.29%	✅
torchvision-densenet121_fp16	32	3,673.82	3,673.03	0.02%	✅
torchvision-inceptionv3	32	1,592.32	1,594.82	-0.16%	✅
torchvision-inceptionv3_fp16	32	2,585.25	2,592.86	-0.29%	✅
cadene-inceptionv4	16	706.98	707.31	-0.05%	✅
cadene-resnext64x4	16	697.85	697.36	0.07%	✅
slim-mobilenet	64	8,354.93	8,344.55	0.12%	✅
slim-nasnetalarge	64	226.73	226.95	-0.10%	✅
slim-resnet50v2	64	2,678.81	2,676.49	0.09%	✅
bert-mrpc-onnx	8	824.56	824.50	0.01%	✅
bert-mrpc-tf	1	387.73	388.61	-0.23%	✅
pytorch-examples-wlang-gru	1	298.53	299.61	-0.36%	✅
pytorch-examples-wlang-lstm	1	316.09	316.43	-0.11%	✅
torchvision-resnet50_1	1	600.12	597.52	0.44%	✅
torchvision-inceptionv3_1	1	334.50	340.06	-1.63%	✅
cadene-dpn92_1	1	395.91	399.86	-0.99%	✅
cadene-resnext101_1	1	325.36	329.54	-1.27%	✅
slim-vgg16_1	1	463.71	463.53	0.04%	✅
slim-mobilenet_1	1	2,069.61	2,046.07	1.15%	✅
slim-inceptionv4_1	1	214.57	214.62	-0.02%	✅
onnx-taau-downsample	1	306.09	307.04	-0.31%	✅
dlrm-criteoterabyte	1	21.70	21.71	-0.05%	✅
dlrm-criteoterabyte_fp16	1	40.74	40.69	0.13%	✅
agentmodel	1	5,782.02	5,783.52	-0.03%	✅
unet_fp16	2	55.95	55.96	-0.01%	✅
resnet50v1_fp16	1	943.66	948.45	-0.50%	✅
bert_base_cased_fp16	64	970.15	970.32	-0.02%	✅
bert_large_uncased_fp16	32	304.83	304.83	0.00%	✅
bert_large_fp16	1	166.78	166.60	0.11%	✅
distilgpt2_fp16	16	1,278.65	1,279.20	-0.04%	✅

This build is OK for merge ✅

migraphx-bot · 2023-09-11T23:59:21Z

:white_check_mark:bert-mrpc-onnx: PASSED: MIGraphX meets tolerance

:white_check_mark:bert-mrpc-tf: PASSED: MIGraphX meets tolerance

:white_check_mark:pytorch-examples-wlang-gru: PASSED: MIGraphX meets tolerance

:white_check_mark:pytorch-examples-wlang-lstm: PASSED: MIGraphX meets tolerance

:white_check_mark:torchvision-resnet50_1: PASSED: MIGraphX meets tolerance

🔴torchvision-inceptionv3_1: FAILED: MIGraphX is not within tolerance - check verbose output

:white_check_mark:cadene-dpn92_1: PASSED: MIGraphX meets tolerance

:white_check_mark:cadene-resnext101_1: PASSED: MIGraphX meets tolerance

:white_check_mark:slim-vgg16_1: PASSED: MIGraphX meets tolerance

:white_check_mark:slim-mobilenet_1: PASSED: MIGraphX meets tolerance

🔴slim-inceptionv4_1: FAILED: MIGraphX is not within tolerance - check verbose output

:white_check_mark:dlrm-criteoterabyte: PASSED: MIGraphX meets tolerance

:white_check_mark:agentmodel: PASSED: MIGraphX meets tolerance

:white_check_mark:unet: PASSED: MIGraphX meets tolerance

:white_check_mark:resnet50v1: PASSED: MIGraphX meets tolerance

🔴bert_base_cased_fp16: FAILED: MIGraphX is not within tolerance - check verbose output

🔴bert_large_uncased_fp16: FAILED: MIGraphX is not within tolerance - check verbose output

:white_check_mark:bert_large: PASSED: MIGraphX meets tolerance

🔴distilgpt2_fp16: FAILED: MIGraphX is not within tolerance - check verbose output

codecov · 2023-09-12T21:41:51Z

Codecov Report

Merging #2167 (7803850) into develop (c399062) will not change coverage.
Report is 1 commits behind head on develop.
The diff coverage is n/a.

❗ Current head 7803850 differs from pull request most recent head 188cf94. Consider uploading reports for the commit 188cf94 to get more accurate results

@@           Coverage Diff            @@
##           develop    #2167   +/-   ##
========================================
  Coverage    91.36%   91.36%           
========================================
  Files          439      439           
  Lines        16493    16493           
========================================
  Hits         15069    15069           
  Misses        1424     1424

pfultz2 · 2023-10-04T17:52:06Z

src/targets/gpu/compile_hip_code_object.cpp

+    if(options.global % options.local != 0 and hip_accept_non_uniform_wg())
+        options.params += " -fno-offload-uniform-block";
+    else
+        assert(options.global % options.local == 0);


This will throw an assert on older rocm that doesnt support the flag. If the flag is not supported than most likely non-uniform blocks is supported by default. I think we can remove this assert.

if flag is not supported then it would have calculated the global WG multiple of local WG. It would work with older rocms as well.
#1976

If the flag is not supported than most likely non-uniform blocks is supported by default

That isn't the case with 5.7. Neither the Flag nor non-uniform blocks are supported.

…MIGraphX into add_fno_uniform

src/targets/gpu/kernels/include/migraphx/kernels/index.hpp

pfultz2 · 2023-10-17T23:07:30Z

@umangyadav Do we have a verify test that uses the non-uniform blocks?

umangyadav · 2023-10-18T00:18:31Z

@umangyadav Do we have a verify test that uses the non-uniform blocks?

yes test_slice2 and roialign tests.

src/targets/gpu/compile_hip_code_object.cpp

atamazov · 2024-03-01T18:09:09Z

@umangyadav Is seems like you have some code that checks if compiler supports some option at runtime. FYI this is how we use check_cxx_compiler_flag() to check the same at build time: ROCm/MIOpen#2741

umangyadav · 2024-03-01T19:28:27Z

we want to check at runtime that's why it is done that way. OTherwise we can also use check_cxx_compiler_flag.

umangyadav requested a review from pfultz2 September 11, 2023 21:24

umangyadav self-assigned this Sep 11, 2023

pfultz2 reviewed Sep 11, 2023

View reviewed changes

src/targets/gpu/CMakeLists.txt Outdated Show resolved Hide resolved

pfultz2 added 2 commits September 11, 2023 18:05

Disable -Wunsafe-buffer-usage when compiling gpu code

7e96807

Format

c0077ed

pfultz2 and others added 4 commits September 12, 2023 12:15

Remove comment

836da11

Add flag to accept non-uniform WG sizes

379fca5

Use same as MIopen

c109d1e

use hip_has_flags

a64ee16

umangyadav force-pushed the add_fno_uniform branch from 653d091 to a64ee16 Compare September 12, 2023 20:57

umangyadav and others added 3 commits September 15, 2023 08:36

Merge branch 'develop' into add_fno_uniform

fa8dc98

Merge branch 'develop' into add_fno_uniform

63d1420

Merge branch 'develop' into add_fno_uniform

bbcb77f

causten added the revisit label Sep 21, 2023

Merge branch 'develop' into add_fno_uniform

4ec969b

pfultz2 reviewed Oct 4, 2023

View reviewed changes

umangyadav and others added 4 commits October 16, 2023 19:35

Fixes for the HIP compute max local

42d8f52

Remove comment

5bf10c4

Merge branch 'develop' into add_fno_uniform

92cc92f

Merge branch 'add_fno_uniform' of github.com:ROCmSoftwarePlatform/AMD…

f3e0cbc

…MIGraphX into add_fno_uniform

umangyadav requested a review from pfultz2 October 16, 2023 23:34

formatting

7f8edb8

pfultz2 reviewed Oct 17, 2023

View reviewed changes

src/targets/gpu/kernels/include/migraphx/kernels/index.hpp Outdated Show resolved Hide resolved

pfultz2 reviewed Oct 17, 2023

View reviewed changes

src/targets/gpu/kernels/include/migraphx/kernels/index.hpp Show resolved Hide resolved

pfultz2 reviewed Oct 17, 2023

View reviewed changes

src/targets/gpu/kernels/include/migraphx/kernels/index.hpp Outdated Show resolved Hide resolved

pfultz2 reviewed Oct 17, 2023

View reviewed changes

src/targets/gpu/kernels/include/migraphx/kernels/index.hpp Outdated Show resolved Hide resolved

Use OCL functions

a207942

umangyadav added 5 commits October 17, 2023 20:04

Works now

0ae8452

Add some comments

a61563e

add back comment

be74a60

Fix

4f3f158

Keep assert

222d6e5

umangyadav requested a review from pfultz2 October 17, 2023 20:17

umangyadav and others added 2 commits October 17, 2023 20:29

Add nolint

477d026

Merge branch 'develop' into add_fno_uniform

fac101b

pfultz2 approved these changes Oct 17, 2023

View reviewed changes

umangyadav requested review from TedThemistokleous and kahmed10 October 18, 2023 00:18

umangyadav removed the revisit label Oct 18, 2023

TedThemistokleous reviewed Oct 18, 2023

View reviewed changes

src/targets/gpu/compile_hip_code_object.cpp Show resolved Hide resolved

TedThemistokleous approved these changes Oct 18, 2023

View reviewed changes

TedThemistokleous added bugfix Fixes a bug found in the code. enhancement New feature or request labels Oct 18, 2023

Merge branch 'develop' into add_fno_uniform

188cf94

causten merged commit 581b1b5 into develop Oct 19, 2023
15 checks passed

causten deleted the add_fno_uniform branch October 19, 2023 14:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add flag to accept non-uniform WG sizes #2167

Add flag to accept non-uniform WG sizes #2167

umangyadav commented Sep 11, 2023 •

edited

Loading

migraphx-bot commented Sep 11, 2023 •

edited

Loading

migraphx-bot commented Sep 11, 2023 •

edited

Loading

codecov bot commented Sep 12, 2023 •

edited

Loading

pfultz2 Oct 4, 2023

umangyadav Oct 4, 2023

umangyadav Oct 4, 2023

pfultz2 commented Oct 17, 2023

umangyadav commented Oct 18, 2023

atamazov commented Mar 1, 2024

umangyadav commented Mar 1, 2024

Add flag to accept non-uniform WG sizes #2167

Add flag to accept non-uniform WG sizes #2167

Conversation

umangyadav commented Sep 11, 2023 • edited Loading

migraphx-bot commented Sep 11, 2023 • edited Loading

migraphx-bot commented Sep 11, 2023 • edited Loading

codecov bot commented Sep 12, 2023 • edited Loading

Codecov Report

pfultz2 Oct 4, 2023

Choose a reason for hiding this comment

umangyadav Oct 4, 2023

Choose a reason for hiding this comment

umangyadav Oct 4, 2023

Choose a reason for hiding this comment

pfultz2 commented Oct 17, 2023

umangyadav commented Oct 18, 2023

atamazov commented Mar 1, 2024

umangyadav commented Mar 1, 2024

umangyadav commented Sep 11, 2023 •

edited

Loading

migraphx-bot commented Sep 11, 2023 •

edited

Loading

migraphx-bot commented Sep 11, 2023 •

edited

Loading

codecov bot commented Sep 12, 2023 •

edited

Loading