Pointwise + Concat fusion

[umangyadav]

Fixes #2735

[migraphx-bot]

Test	Batch	Rate new cc8ef6	Rate old 098fc8	Diff	Compare
torchvision-resnet50	64	2,861.93	2,851.64	0.36%	✅
torchvision-resnet50_fp16	64	6,597.07	6,515.86	1.25%	✅
torchvision-densenet121	32	2,103.63	2,091.27	0.59%	✅
torchvision-densenet121_fp16	32	3,693.95	3,690.48	0.09%	✅
torchvision-inceptionv3	32	1,604.32	1,603.09	0.08%	✅
torchvision-inceptionv3_fp16	32	2,578.92	2,573.96	0.19%	✅
cadene-inceptionv4	16	726.93	724.83	0.29%	✅
cadene-resnext64x4	16	683.14	683.04	0.01%	✅
slim-mobilenet	64	5,946.30	5,946.24	0.00%	✅
slim-nasnetalarge	64	153.98	nan	nan%	❌
slim-resnet50v2	64	2,663.43	2,670.54	-0.27%	✅
bert-mrpc-onnx	8	917.80	917.66	0.01%	✅
bert-mrpc-tf	1	433.88	437.34	-0.79%	✅
pytorch-examples-wlang-gru	1	423.35	425.95	-0.61%	✅
pytorch-examples-wlang-lstm	1	384.47	389.34	-1.25%	✅
torchvision-resnet50_1	1	604.96	607.99	-0.50%	✅
cadene-dpn92_1	1	392.57	392.28	0.07%	✅
cadene-resnext101_1	1	332.15	332.16	-0.00%	✅
onnx-taau-downsample	1	305.77	305.63	0.05%	✅
dlrm-criteoterabyte	1	28.78	28.76	0.08%	✅
dlrm-criteoterabyte_fp16	1	49.65	49.58	0.13%	✅
agentmodel	1	7,655.27	6,297.71	21.56%	🔆
unet_fp16	2	57.56	57.61	-0.07%	✅
resnet50v1_fp16	1	888.54	875.33	1.51%	✅
resnet50v1_int8	1	795.67	787.14	1.08%	✅
bert_base_cased_fp16	64	1,055.43	1,055.34	0.01%	✅
bert_large_uncased_fp16	32	311.93	311.86	0.02%	✅
bert_large_fp16	1	159.26	159.29	-0.02%	✅
distilgpt2_fp16	16	1,858.49	1,857.62	0.05%	✅
yolov5s	1	467.33	484.28	-3.50%	🔴
tinyllama	1	32.88	32.84	0.11%	✅
vicuna-fastchat	1	159.38	159.75	-0.23%	✅
whisper-tiny-encoder	1	348.79	348.21	0.17%	✅
whisper-tiny-decoder	1	397.67	398.85	-0.30%	✅

This build is not recommended to merge 🔴

[migraphx-bot]

     ✅ bert-mrpc-onnx: PASSED: MIGraphX meets tolerance

     ✅ bert-mrpc-tf: PASSED: MIGraphX meets tolerance

     ✅ pytorch-examples-wlang-gru: PASSED: MIGraphX meets tolerance

     ✅ pytorch-examples-wlang-lstm: PASSED: MIGraphX meets tolerance

     ✅ torchvision-resnet50_1: PASSED: MIGraphX meets tolerance

     ✅ cadene-dpn92_1: PASSED: MIGraphX meets tolerance

     ✅ cadene-resnext101_1: PASSED: MIGraphX meets tolerance

     ✅ dlrm-criteoterabyte: PASSED: MIGraphX meets tolerance

     ✅ agentmodel: PASSED: MIGraphX meets tolerance

     ✅ unet: PASSED: MIGraphX meets tolerance

     ✅ resnet50v1: PASSED: MIGraphX meets tolerance

     ✅ bert_base_cased_fp16: PASSED: MIGraphX meets tolerance

🔴bert_large_uncased_fp16: FAILED: MIGraphX is not within tolerance - check verbose output

     ✅ bert_large: PASSED: MIGraphX meets tolerance

     ✅ yolov5s: PASSED: MIGraphX meets tolerance

     ✅ tinyllama: PASSED: MIGraphX meets tolerance

     ✅ vicuna-fastchat: PASSED: MIGraphX meets tolerance

     ✅ whisper-tiny-encoder: PASSED: MIGraphX meets tolerance

     ✅ whisper-tiny-decoder: PASSED: MIGraphX meets tolerance

     ✅ distilgpt2_fp16: PASSED: MIGraphX meets tolerance

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.83%. Comparing base (9f02b19) to head (e5fd209).
Report is 2 commits behind head on develop.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #2785      +/-   ##
===========================================
+ Coverage    91.82%   91.83%   +0.01%     
===========================================
  Files          477      477              
  Lines        18112    18146      +34     
===========================================
+ Hits         16631    16665      +34     
  Misses        1481     1481              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[pfultz2]

Why is this prefixed with a !? I would prefer not to have it start with a special character because this will become _x0 in the C++.

[umangyadav]

I wasn't sure. I copied same logic from pointwise_concat_pointwise fusion. I can remove it.

[pfultz2]

It uses x0 in that pass.

[umangyadav]

AMDMIGraphX/src/fuse_concat.cpp

Line 158 in b1a24e5

auto param = rm->add_parameter("!" + concat_param_name, concat_param->get_shape());

I am referring this line.

[pfultz2]

We should probably only do this input fusion if the number of noops is low. I would use something like max_noops = max(1, concat_ins->inputs().size() / 4) to calculate the max number of noops we allow.

[umangyadav]

We should probably only do this input fusion if the number of noops is low. I would use something like max_noops = min(1, concat_ins->inputs().size() / 4) to calculate the max number of noops we allow.

This would simply mean allowing at max 1 no-op.

Added that rule and a test.