Loop perfomance

[xeeetu]

Hello, I want to know which parameters can be derived from the loop and which can't (can I take memoryInfo out of the loop?), can the loop only call session.Run? If you give me a link to some example, I would appreciate it.

That's what I do every iteration:

float *blob = nullptr;
std::vector<int64_t> inputTensorShape {1, 3, int(this->inputImageShape.width), int(this->inputImageShape.height)};
this->preprocessing(image, blob, inputTensorShape);

size_t inputTensorSize = utils::vectorProduct(inputTensorShape);

std::vector<float> inputTensorValues(blob, blob + inputTensorSize);

std::vector<Ort::Value> inputTensors;

Ort::MemoryInfo memoryInfo = Ort::MemoryInfo::CreateCpu(
        OrtAllocatorType::OrtArenaAllocator, OrtMemType::OrtMemTypeDefault);

inputTensors.push_back(Ort::Value::CreateTensor<float>(
        memoryInfo, inputTensorValues.data(), inputTensorSize,
        inputTensorShape.data(), inputTensorShape.size()
));

std::vector<Ort::Value> outputTensors = this->session.Run(Ort::RunOptions{nullptr},
                                                         _inputNodeNames.data(),
                                                          inputTensors.data(),
                                                          1,
                                                          _outputNodeNames.data(),
                                                          1);

cv::Size resizedShape = cv::Size((int)inputTensorShape[3], (int)inputTensorShape[2]);
std::vector<Detection> result = this->postprocessing(resizedShape,
                                                     image.size(),
                                                     outputTensors,
                                                     confThreshold, iouThreshold);

delete[] blob;