Java: how long should I use a session

[huehnerlady]

Hi,

currently we create a session for every execution of our onnx model, which we already read is not advised.
Unfortunately we cannot find anything in the documentation about when we should create such a session.
Should we do that at the initialisation of our executor, so the session is open the whole time our java-process is running, which can be weeks? Or is there another advised process?

[Craigacp]

The session object owns the model and should be a long lived object. If your JVM is up for weeks and you need inferences constantly during that time I don't see a reason not to keep the session up. If your inference needs are bursty then you can bring up a session and tear it down when the burst passes.