Rework template to better use MCP capabilities

[avishniakov]

Describe changes

I restructured the example a bit to make it more MCP-focused. Some steps got aggregated in the promotion stage to do promotion in MCP and MR in one shot, not stressing much on MR. Also, MLFlow deployment in inference is pulled into the prediction step.
This PR also introduces the Deployment Pipeline as an extra layer being nicely glued together by the Model Control Plane. Due to the fact, that we are a bit limited on deployers flexibility at the moment - we only really deploy a service on the local orchestrator using MLFlow deployer, otherwise model will be loaded to memory on inference, and the deployment pipeline becomes kinda dummy then.
New flow:

README is up-to-date, but with draft ugly pictures - to be done still, if we confirm the approach

Before:

    ########## Promotion stage ##########
    latest_version, current_version = promote_get_versions(
        after=["mlflow_register_model_step"],
    )
    latest_deployment = mlflow_model_registry_deployer_step(
        id="deploy_latest_model_version",
        registry_model_name=pipeline_extra["mlflow_model_name"],
        registry_model_version=latest_version,
        replace_existing=True,
    )
    latest_metric = promote_get_metric(
        id="get_metrics_latest_model_version",
        dataset_tst=dataset_tst,
        deployment_service=latest_deployment,
    )

    current_deployment = mlflow_model_registry_deployer_step(
        id="deploy_current_model_version",
        registry_model_name=pipeline_extra["mlflow_model_name"],
        registry_model_version=current_version,
        replace_existing=True,
        after=["get_metrics_latest_model_version"],
    )
    current_metric = promote_get_metric(
        id="get_metrics_current_model_version",
        dataset_tst=dataset_tst,
        deployment_service=current_deployment,
    )

    (
        was_promoted,
        promoted_version,
    ) = promote_metric_compare_promoter_in_model_registry(
        latest_metric=latest_metric,
        current_metric=current_metric,
        latest_version=latest_version,
        current_version=current_version,
    )
    promote_model_version_in_model_control_plane(was_promoted)

After

    ########## Promotion stage ##########
    (
        latest_metric,
        current_metric,
    ) = compute_performance_metrics_on_current_data(
        dataset_tst=dataset_tst, after=["model_evaluator"]
    )

    promote_with_metric_compare(
        latest_metric=latest_metric,
        current_metric=current_metric,
    )
    last_step = "promote_with_metric_compare"

Also, I used MCP capabilities to keep track of the model registry version and to demonstrate how we can glue things together with artifact metadata.

# in step
    log_artifact_metadata(
        output_name="model",
        model_registry_version=Client()
        .active_stack.model_registry.list_model_versions(name=name)[-1]
        .version,
    )

# fetch later
def _get_mr_version(model_version: ModelVersionResponseModel) -> str:
    return model_version.get_model_object("model").metadata["model_registry_version"].value

Pre-requisites

Please ensure you have done the following:

• If my change requires a change to docs, I have updated the documentation accordingly.
• I have added tests to cover my changes.
• I have updated ref-zenml in .github/workflows/ci.yml accordingly (if you don't know - main would be a solid choice)

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Other (add details above)

[avishniakov]

@htahir1 @strickvl this PR is now in its' final state, please proceed reviewing it, so we can publish it instead of previous under same tag

[github-actions]

Images automagically compressed by Calibre's image-actions ✨

Compression reduced images by 37.8%, saving 836.38 KB.

Filename	Before	After	Improvement	Visual comparison
assets/01_etl.png	71.90 KB	63.63 KB	-11.5%	View diff
assets/02_hp.png	82.21 KB	71.11 KB	-13.5%	View diff
assets/03_train.png	26.66 KB	20.74 KB	-22.2%	View diff
assets/04_promotion.png	60.83 KB	47.89 KB	-21.3%	View diff
assets/05_deployment.png	45.02 KB	35.80 KB	-20.5%	View diff
assets/06_batch_inference.png	180.60 KB	162.08 KB	-10.3%	View diff
template/.assets/00_pipelines_composition.png	1.70 MB	975.01 KB	-44.1%	View diff