Skip to content
This repository has been archived by the owner on Oct 16, 2024. It is now read-only.

GTO: Document SemVer practices for ML models #231

Open
aguschin opened this issue Nov 22, 2022 · 0 comments
Open

GTO: Document SemVer practices for ML models #231

aguschin opened this issue Nov 22, 2022 · 0 comments
Labels
A: docs Area: user documentation (gatsby-theme-iterative) p2-nice-to-have Less of a priority at the moment. We don't usually deal with this immediately. type: discussion

Comments

@aguschin
Copy link
Contributor

aguschin commented Nov 22, 2022

Semantic versioning is the accepted way to version code. How should artifacts be versioned?
I have been asked this by a Data Scientist some time ago. Given that everyone is free to do whatever he wants, perhaps giving a hint is not bad...?

I formulated a reasonable convention for models, not sure if it could be of any use:

Patch

Model as a black-box is as before, it only outputs different numbers.

Typical scenario: model have been trained with more recent data
Typical scenario 2: changed hyper-parameters

Minor

May want to take advantage of additional outputs or additional functionalities

Typical scenario 1: model now has predict_proba() in addition to predict()
Typical scenario 2: model now outputs a json with an additional field confidence_interval, in addition to predicted_values

Major

Need to re-visit the code that calls the model to serve it (breaking change)

Typical scenario 1: model APIs have changed
Typical scenario 2: model expects different input data format
Typical scenario 3: model relies on different libraries, need to re-build the venv (or even the OS-level libraries)

Originally posted by @francesco086 in #199 (comment)

🧵 See the thread for more opinions on this

@aguschin aguschin changed the title Docs: add a page about SemVer for ML Docs: explain SemVer for ML Nov 22, 2022
@aguschin aguschin added the A: docs Area: user documentation (gatsby-theme-iterative) label Nov 22, 2022
@aguschin aguschin mentioned this issue Nov 23, 2022
@jorgeorpinel jorgeorpinel changed the title Docs: explain SemVer for ML GTO: explain SemVer for ML Nov 25, 2022
@omesser omesser added the p2-nice-to-have Less of a priority at the moment. We don't usually deal with this immediately. label Mar 6, 2023
@omesser omesser changed the title GTO: explain SemVer for ML GTO: explain SemVer for ML models Mar 6, 2023
@omesser omesser changed the title GTO: explain SemVer for ML models GTO: Document SemVer practices for ML models Mar 6, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
A: docs Area: user documentation (gatsby-theme-iterative) p2-nice-to-have Less of a priority at the moment. We don't usually deal with this immediately. type: discussion
Projects
None yet
Development

No branches or pull requests

3 participants