Replies: 3 comments 1 reply
-
You've identified most of the functions that are used internally. This is going to be a lot of work, plus you'll have to do this in a fork and maintain the fork since it wouldn't make sense for us to accept this contribution as ORT is very tightly coupled with the ONNX model format which is based on protobuf. If you don't care about the size of the build you can skip the ORT_MINIMAL_BUILD related ifdefs. |
Beta Was this translation helpful? Give feedback.
-
Thanks @pranavsharma for the advice! We dropped that idea and are instead targeting directly using ORT model files. We have 2 questions about these:
This is the pseudo-code of what we tried for this:
That code works as long as we keep everything on the CPU (i.e, if we do not use
Thanks! |
Beta Was this translation helpful? Give feedback.
-
Then I believe there is a bug in the ONNX2ORT conversion when setting Device=DirectML. I opened a GitHub issue with all the information in here: |
Beta Was this translation helpful? Give feedback.
-
I have a class called MyModelProto that contains all the same fields that an ONNX ModelProto structure would have (but without any Protobuf dependencies). My goal is to run ORT with my 3rd party MyModelProto (already loaded on RAM) rather than with the Protobuf-based onnx::ModelProto (which ORT reads from the onnx file path).
Looking at the code, I wanted to find the best approach to implement this load. I believe that using/modifying an existing load function should be the easiest approach? (And if so, which one(s)?)
In inference_session.h, I see there are many load functions for the ONNX and ORT format models. From what I understand, the onnx model is loaded by passing a protobuf object corresponding to the model file and then having this “model_proto” be copied by the API.
Then, it seems that several additional files call onnx functions directly in order to get the data stored inside AttributeProto, TensorProto, etc…
Files/Functions:
I guess I will need to modify several of the functions in the above files in order to read directly from my MyModelProto? I see that ORT has a class Model, which contains the fields for the graph and basic information, such as model version, producer version, ir version, doc_string, etc.
For simplicity, an overview of my questions with some additional ones:
Beta Was this translation helpful? Give feedback.
All reactions