Do stored Orange models (.pkcl) also store processing?

[TonciG]

Hi,

My understanding is that Orange's Save Model widget saves, among other things, values for the parameters and hyperparameters of a trained model.

What happens if the model also uses Preprocessor widget as one of its inputs (please see image attached)? Will the preprocessing methods, together with their parameters (e.g. min and max if Normalize [0,1] is used), also be part of .pkcls file?

If yes, then is it possible to use such model (via Load Model widget) on a not previously pre-processed data (e.g. test data or data for which we want to make predictions)? Would the model in .pkcls file apply the pre-processing methods (and their appropriate parameters) before it makes predictions with the data?

Thanks

[markotoplak]

Yes, all models in Orange are supposed to save their preprocessing, so they should work on a separate (non-processed) test set, if its features match.

This is implemented as models storing the domain of the data on its input. The domain in turn stores all previous transformations. Then, when a model is given (test) data to predict, the data is first converted into the domain where the model was learned, which in turn applies processing.

[TonciG]

Thanks on your response.

So the only condition for this to work on a new (non-preprocessed) data is that the features between the two datasets must match?

[markotoplak]

Yes. More precisely, for two Orange Tables the features of the test data need to match (with name and meaning) those of your train data prior to preprocessing.

[TonciG]

Thanks!

[TonciG]

Hi Marko,
What you suggest above seems to work only in Orange environment.
If you try to use an Orange model to make prediction on an unseen (and non pre-processed) data in a standalone Python code via .pkcls file, the automatic pre-processing does not work. To make it work, you first need to manually pre-process data before you feed it to the model.
Tonci

[markotoplak]

It should also work from scripts IF your data is set the same way (meaning in Orange.data.Table).

[markotoplak]

... And you are calling the model with the data Table. If you call predict with a numpy array, you lose the table information and all these transformations because you lost the domain.

So something like predictions = model(datatable) should keep all transforms (where model is your unpicled model).

[TonciG]

predict() does not, which is understandable as you explained.
Regarding model(datatable), it works but lists a number of warnings. Please see figure attached, which shows both the code and console output.

[markotoplak]

Yup, it is what it says, you are using different (dependency) versions between your two installations. I'd ignore this unless you have problems. If you have them, manually install a specific version of scikit-learn in one of the environments.

[TonciG]

So the warning is because sklearn used by Orange 3 is different to the one in my virtual environment?

[markotoplak]

Yes.

[TonciG]

Thanks, and sorry for pestering you!

P.S. Lijep pozdrav!

[VanBuren77]

How can I extract the preprocessing functions from a pickled model?

[markotoplak]

What do you mean by "extract"? These are only meant to be applied, for example, with the Apply Domain widget.

Introspection is difficult. Each variable that does preprocessing has a compute_value. compute_value is a function that takes some data and outputs the results of the conversion.

Some docs on derived variables: https://orange3.readthedocs.io/projects/orange-data-mining-library/en/latest/reference/data.variable.html

The way each compute_value stores its parameters is undefined so to actually inspect them, you will need to inspect the type first and then act accordingly.