Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(mlflow): Update Mlflow, API server, pyfunc server and batch predictor #617

Merged
merged 12 commits into from
Nov 18, 2024

Conversation

deadlycoconuts
Copy link
Contributor

@deadlycoconuts deadlycoconuts commented Nov 14, 2024

Description

This PR is the second part (2 out of 3) of a series of PRs to update the version of Mlflow used by Merlin (to 1.26.1*). These changes are:

  1. Update the version of Mlflow used in the Merlin SDK and publish it to PyPI
  2. Update
    • the Merlin pyfunc server and batch predictor to use the updated Merlin SDK version released in step 1 and publish them to PyPI
    • the Merlin API server to ensure that requests sent to Mlflow reflect the update API endpoint contracts
    • the Mlflow image so that it's built using version 1.26.1
  3. Update the default pyfunc server and batch predictor version in the Merlin SDK and publish its new version to PyPI

Main Modifications

  • .github/workflows/external.yml - Updated the CICD job to release a new version of the Mlflow image
  • api/mlflow/mlflow.go - Added a header in all requests sent to the Mlflow server from the Merlin API server because it's now required
  • api/mlflow/response.go - Changed the data type of certain fields in Mlflow's responses sent to Merlin
  • mlflow/Dockerfile - Updated Dockerfile of the Mlflow server with newer dependencies
  • python/batch-predictor/requirements.txt - Updated the requirements file with a newer version of the SDK which requires version 1.26.1 of Mlflow
  • python/pyfunc-server/requirements.txt - Updated the requirements file with a newer version of the SDK which requires version 1.26.1 of Mlflow

Tests

Checklist

  • Added PR label
  • Added unit test, integration, and/or e2e tests
  • Tested locally
  • Updated documentation
  • Update Swagger spec if the PR introduce API changes
  • Regenerated Golang and Python client if the PR introduces API changes

Release Notes

NONE

@deadlycoconuts deadlycoconuts added the enhancement New feature or request label Nov 14, 2024
@deadlycoconuts deadlycoconuts self-assigned this Nov 14, 2024
@deadlycoconuts deadlycoconuts force-pushed the update_mlflow_in_non_sdk_components branch from 8690f51 to 26131d9 Compare November 15, 2024 05:29
@deadlycoconuts deadlycoconuts force-pushed the update_mlflow_in_non_sdk_components branch from 26131d9 to d7b4e48 Compare November 15, 2024 05:37
@deadlycoconuts deadlycoconuts force-pushed the update_mlflow_in_non_sdk_components branch 7 times, most recently from 2325428 to 7d0330d Compare November 15, 2024 18:52
@deadlycoconuts deadlycoconuts force-pushed the update_mlflow_in_non_sdk_components branch from 7680aba to 5a4e9fc Compare November 16, 2024 15:28
@deadlycoconuts deadlycoconuts marked this pull request as ready for review November 16, 2024 15:45
Copy link
Contributor

@shydefoo shydefoo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the detailed notes in the PR explaining each step.

mlflow/Dockerfile Show resolved Hide resolved
@deadlycoconuts
Copy link
Contributor Author

Thanks a lot for the quick review! :D I'll merge this soon!

@deadlycoconuts deadlycoconuts merged commit eefea9e into main Nov 18, 2024
33 checks passed
@deadlycoconuts deadlycoconuts deleted the update_mlflow_in_non_sdk_components branch November 18, 2024 06:58
deadlycoconuts added a commit that referenced this pull request Nov 18, 2024
…618)

# Description
This PR is the **third part** (3 out of 3) of a series of PRs to update
the version of Mlflow used by Merlin (to `1.26.1`*). These changes are:

1. Update the version of Mlflow used in the Merlin SDK and publish it to
PyPI
2. Update 
- the Merlin pyfunc server and batch predictor to use the updated Merlin
SDK version released in step 1 and publish them to PyPI
- the Merlin API server to ensure that requests sent to Mlflow reflect
the update API endpoint contracts
   - the Mlflow image so that it's built using version `1.26.1`
3. Update the default pyfunc server and batch predictor version in the
Merlin SDK and publish its new version to PyPI

Since the pyfunc server and batch predictor versions have already been
updated in the PR #617, we need to ensure that the SDK is able to use
these newly released versions by default.

# Modifications
- `python/sdk/merlin/model.py` - Update the version bounds of the
default pyfunc server and batch predictor version in the SDK

# Tests
<!-- Besides the existing / updated automated tests, what specific
scenarios should be tested? Consider the backward compatibility of the
changes, whether corner cases are covered, etc. Please describe the
tests and check the ones that have been completed. Eg:
- [x] Deploying new and existing standard models
- [ ] Deploying PyFunc models
-->

# Checklist
- [x] Added PR label
- [ ] Added unit test, integration, and/or e2e tests
- [x] Tested locally
- [ ] Updated documentation
- [ ] Update Swagger spec if the PR introduce API changes
- [ ] Regenerated Golang and Python client if the PR introduces API
changes

# Release Notes
```release-note
NONE
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants