--Batch_Size does not work since 1 onnx model just supports 1 fixed batchsize

[fsword73]

The command line:

plaidbench --batch-size 16 --result .\shuffle_results onnx --plaid resnet50

in frontend_onnx.py line 134, Add 1 more funciton:
print(self.x[:self.params.batch_size].shape)
partial_result = self.rep.run([self.x[:self.params.batch_size]])

The output is always (1, 3, 224, 224) for self.x[:self.params.batch_size].shape.

Go-into test_data_0.npz. There is only 1 image with batchsize 1.

I debug this issue since I noticed that Batch_Size 1,2,4,8,16,32 is linear time.

I refered to ONNX issue (Reshape Input for Dynamic Batch Size). 1 ONNX model just works for 1 batch_size. In other words, If I would to try batchSize 1, 2, 4,8,16,32, 64, I need 6 different ONNX model.

I wrote follwing test to verify it :

import onnx
import onnx_plaidml.backend
import numpy as np
import datetime

# Assumption: test_images.npz contains a tensor, "inputs", whose
# most-major dimension strides through images to be supplied to the model.
for i in ([1,2,4,8,16,32,64]):
dimensions = (i, 3, 224, 224)
batch32 = (np.random.rand(*dimensions) + 1.) / 52.
print (batch32.shape)

model = onnx.load('resnet50_model.onnx')
rep = onnx_plaidml.backend.prepare(model, None)
output = rep.run([batch32])

PS D:\JianYang\Testing\plaidvision> python.exe .\test_onnx2.py
(1, 3, 224, 224)
INFO:plaidml:Opening device "carrizo.0"
INFO:plaidml:Analyzing Ops: 580 of 917 operations complete
WARNING:plaidml:Reached scheduler optimization timeout
(2, 3, 224, 224)
INFO:plaidml:Opening device "carrizo.0"
INFO:plaidml:Analyzing Ops: 477 of 918 operations complete
Traceback (most recent call last):
File ".\test_onnx2.py", line 15, in
output = rep.run([batch32])
File "C:\Python36\lib\site-packages\onnx_plaidml\backend.py", line 146, in run
self.invoker.invoke()
File "C:\Python36\lib\site-packages\plaidml_init.py", line 1426, in invoke
return Invocation(self.ctx, self)
File "C:\Python36\lib\site-packages\plaidml_init.py", line 1432, in init
self.as_parameter = lib().plaidml_schedule_invocation(ctx, invoker)
File "C:\Python36\lib\site-packages\plaidml_init.py", line 716, in _check_err
self.raise_last_status()
File "C:\Python36\lib\site-packages\plaidml\library.py", line 131, in raise_last_status
raise self.last_status()
plaidml.exceptions.Unknown: Invalid reshape