Model save to File

[nMaroulis]

Hi and congratulations for this great work. My question is if it's possible to save to a file the Tensorflow model in order to load it and use it for real-time inference?
Thanks

[wkirgsn]

Hey, it is definitely possible. That is also true for Pytorch.
See e.g. https://www.tensorflow.org/tutorials/keras/save_and_load

Having said that, I am currently working on a tensorflow refactor, which will showcase the usage of TNNs with tensorflow in a very similar way as it is for Pytorch now. Stay tuned!

[nMaroulis]

Thanks for your quick reply Wilhem, I'm using the new tf2 notebook and the optimization has been a lot easier, thanks. My problem with saving the model persists. I added the following decorator above the TNNModel class "@tf.keras.saving.register_keras_serializable" since it was mandatory, but then when I load the model I get the following error:

"ValueError: You cannot build your model by calling build if your layers do not support float type inputs. Instead, in order to instantiate and build your model, call your model on real tensor data (of the correct dtype).

The actual error from call is: Exception encountered when calling layer 'rnn_2' (type RNN).

TNNCell.get_initial_state() got multiple values for argument 'inputs'

Call arguments received by layer 'rnn_2' (type RNN):
• inputs=tf.Tensor(shape=(180, 15, 15), dtype=float32)
• mask=None
• training=None
• initial_state=None
• constants=None."

Any comments?

Thanks again.

[wkirgsn]

Skimming this section https://www.tensorflow.org/tutorials/keras/save_and_load#saving_custom_objects
I also guess you need to implement get_config and from_config in addition to the decorator.

Also, maybe try removing the "get_initial_state" function in the Model, as this seems to be the problematic part, and it is not used by the framework as far as I can oversee it. I think this is a function, that is called by other Tensorflow abstractions that are not relevant in this notebook.

[Fytron]

Hello. I have been facing the same issue trying to save and load the model in PyTorch or ONNX format.

In Tensorflow I get:
model.save("tf_model", save_format="tf")

WARNING:absl:Found untraced functions such as _update_step_xla while saving (showing 1 of 1). These functions will not be directly callable after loading.
INFO:tensorflow:Assets written to: tf_model\assets
INFO:tensorflow:Assets written to: tf_model\assets

Using the following helps to load the model again though:
custom_objects = {
"TNNCell": TNNCell,
"TNNModel": TNNModel,
}
model = load_model('tf_model', custom_objects=custom_objects)

It also seems like ONNX is not able to capture the complete model and I am not sure what to do.

[wkirgsn]

I guess I will open up an issue for tensorflow model saving and loading, since this seems to be too nontrivial

[wkirgsn]

opened #5 for this

[wkirgsn]

#5 adds examples for model saving