02-sagemaker-containers.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "e6b39bb5-1408-45ab-af98-fa786ce6d55f",
   "metadata": {},
   "source": [
    "# Step 2: Add SageMaker processing and training jobs\n",
    "In this step you move data processing and model training into [SageMaker Docker containers](https://docs.aws.amazon.com/sagemaker/latest/dg/docker-containers.html) and use [SageMaker Python SDK](https://sagemaker.readthedocs.io/en/stable/index.html) to interact with SageMaker.\n",
    "\n",
    "![](img/six-steps-2.png)\n",
    "\n",
    "SageMaker makes use of Docker containers to enable developers to process data, train and deploy models. Containers allow developers and data scientists to package software into standardized units that run consistently on any platform that supports Docker. Containers ensure that code, runtime, system tools, system libraries, and settings are all in the same place, isolating them from the execution environment. It guarantees a consistent runtime experience regardless of where a container is being run.\n",
    "\n",
    "SageMaker also provides pre-build containers with popular data processing frameworks and ML algorithms. All SageMaker built-in algorithms are delivered as Docker containers."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ffd00ea7",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\"> Make sure you using <code>Python 3</code> kernel in JupyterLab for this notebook.</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "1c357f9a-0df7-4358-a231-62aaaeec605f",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Stored variables and their in-db values:\n",
      "baseline_s3_url                         -> 's3://sagemaker-us-east-1-906545278380/from-idea-t\n",
      "bucket_name                             -> 'sagemaker-us-east-1-906545278380'\n",
      "bucket_prefix                           -> 'from-idea-to-prod/xgboost'\n",
      "dataset_file_local_path                 -> 'data/bank-additional/bank-additional-full.csv'\n",
      "domain_id                               -> 'd-igloxuzrs3z2'\n",
      "evaluation_s3_url                       -> 's3://sagemaker-us-east-1-906545278380/model-regis\n",
      "experiment_name                         -> 'from-idea-to-prod-experiment-07-11-05-31'\n",
      "initialized                             -> True\n",
      "input_s3_url                            -> 's3://sagemaker-us-east-1-906545278380/from-idea-t\n",
      "mlflow_arn                              -> 'arn:aws:sagemaker:us-east-1:906545278380:mlflow-t\n",
      "mlflow_name                             -> 'mlflow-d-igloxuzrs3z2'\n",
      "model_package_group_name                -> 'model-registry-patterns-pipeline-model-10-19-47-1\n",
      "output_s3_url                           -> 's3://sagemaker-us-east-1-906545278380/from-idea-t\n",
      "pipeline_name                           -> 'model-registry-patterns-pipeline-10-19-47-18'\n",
      "prediction_baseline_s3_url              -> 's3://sagemaker-us-east-1-906545278380/model-regis\n",
      "project                                 -> 'model-registry-patterns'\n",
      "region                                  -> 'us-east-1'\n",
      "sm_model_package_group_name             -> 'model-registry-patterns-pipeline-model-10-19-47-1\n",
      "sm_role                                 -> 'arn:aws:iam::906545278380:role/mlops-workshop-dom\n",
      "space_name                              -> 'jupyterlab-space'\n",
      "target_col                              -> 'y'\n",
      "test_s3_url                             -> 's3://sagemaker-us-east-1-906545278380/from-idea-t\n",
      "train_s3_url                            -> 's3://sagemaker-us-east-1-906545278380/from-idea-t\n",
      "training_job_name                       -> 'from-idea-to-prod-training-2024-10-31-11-09-18-05\n",
      "user_profile_name                       -> 'studio-user-fb6d0370'\n",
      "validation_s3_url                       -> 's3://sagemaker-us-east-1-906545278380/from-idea-t\n"
     ]
    }
   ],
   "source": [
    "%store -r \n",
    "\n",
    "%store\n",
    "\n",
    "try:\n",
    "    initialized\n",
    "except NameError:\n",
    "    print(\"+++++++++++++++++++++++++++++++++++++++++++++++++\")\n",
    "    print(\"[ERROR] YOU HAVE TO RUN 00-start-here notebook   \")\n",
    "    print(\"+++++++++++++++++++++++++++++++++++++++++++++++++\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "f6e4e390-2278-4a1c-8886-13d67507c818",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'2.227.0'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import time\n",
    "import boto3\n",
    "import botocore\n",
    "import numpy as np  \n",
    "import pandas as pd  \n",
    "import sagemaker\n",
    "import os\n",
    "import mlflow\n",
    "from time import gmtime, strftime, sleep\n",
    "from sagemaker.processing import FrameworkProcessor, ProcessingInput, ProcessingOutput\n",
    "from sagemaker.sklearn.estimator import SKLearn\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from mlflow import MlflowClient\n",
    "from IPython.display import Javascript\n",
    "\n",
    "sagemaker.__version__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "5214e551-4800-47c2-9eae-337f4fc5ed7f",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "session = sagemaker.Session()\n",
    "sm = session.sagemaker_client\n",
    "training_job_name = None"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d5beedac-1847-47eb-b294-5eac2ecaa7ef",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "## Prepare an MLflow experiment\n",
    "Re-use the existing experiment we created in the previous notebook. You're going to track new runs in the same experiment.\n",
    "You can also create a new experiment to track runs in this notebook – in this case just uncomment the following code cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2d71279f-dca1-4c0f-b073-00f0b5ebf0e5",
   "metadata": {},
   "outputs": [],
   "source": [
    "experiment_suffix = strftime('%d-%H-%M-%S', gmtime())\n",
    "registered_model_name = f\"from-idea-to-prod-job-model-{experiment_suffix}\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9e798262-7ee3-45e8-86b0-da6ff1cf9a61",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Uncomment code block (Cmd + /) if you would like to create a new experiment\n",
    "\n",
    "# experiment_name = f\"from-idea-to-prod-experiment-{experiment_suffix}\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c2737ea1-3774-456b-9601-c6b173736340",
   "metadata": {},
   "outputs": [],
   "source": [
    "mlflow.set_tracking_uri(mlflow_arn)\n",
    "experiment = mlflow.set_experiment(experiment_name=experiment_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2b39c88c-6aed-47b4-8048-a27f35444e10",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "## Process data with SageMaker processing jobs\n",
    "Use [SageMaker Processing](https://docs.aws.amazon.com/sagemaker/latest/dg/processing-job.html) by simply providing a Python data preprocessing script and choosing a [SageMaker SDK processor](https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_processing.html) class.\n",
    "You must upload the input data to S3 and specify an S3 location for output data. SageMaker Processing automatically loads the input data from S3 and uploads transformed data back to S3 when the job is complete. The processing container image can either be an Amazon SageMaker built-in image or a custom image that you provide. The underlying infrastructure for a Processing job is fully managed by Amazon SageMaker. Cluster resources are provisioned for the duration of your job, and cleaned up when a job completes.\n",
    "\n",
    "![](img/sagemaker-processing.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "31b4f1fe-ef68-4151-8d7d-28269e39b68a",
   "metadata": {},
   "source": [
    "Your input data must be stored in an Amazon S3 bucket. Alternatively, you can use [Amazon Athena](https://sagemaker.readthedocs.io/en/stable/api/utility/inputs.html#sagemaker.dataset_definition.inputs.AthenaDatasetDefinition) or [Amazon Redshift](https://sagemaker.readthedocs.io/en/stable/api/utility/inputs.html#sagemaker.dataset_definition.inputs.AthenaDatasetDefinition) as input sources.\n",
    "\n",
    "Upload the input dataset to an Amazon S3 bucket:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "01ba50c3-8428-4756-9a2c-8b6f8d19a52b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "input_s3_url = session.upload_data(\n",
    "    path=\"data/bank-additional/bank-additional-full.csv\",\n",
    "    bucket=bucket_name,\n",
    "    key_prefix=f\"{bucket_prefix}/input\"\n",
    ")\n",
    "\n",
    "%store input_s3_url"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "118099a1-e789-435e-a309-4dd9043f219d",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# !aws s3 ls {bucket_name}/{bucket_prefix} --recursive"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8172a970-448a-4684-81b0-dc01978112d1",
   "metadata": {},
   "source": [
    "### Run processing script in a SageMaker Framework container\n",
    "You start with running your data processing script in a SageMaker pre-built managed container."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7babd5da-715d-47e7-9a5e-fa88296e8af7",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\"> Since you're using MLflow in the processing script, you need to provide a <code>requirements.txt</code> so that the container installs mlflow and MLflow SageMaker plugin before running the processing script.</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "06f231dc-e97c-4cf7-b81a-8f415ac77c14",
   "metadata": {},
   "outputs": [],
   "source": [
    "dependencies_dir=\"./processing/requirements/\"\n",
    "%mkdir -p processing\n",
    "%mkdir -p {dependencies_dir}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0f08051e-8c64-42d7-9b74-24db11146e0d",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile {dependencies_dir}requirements.txt\n",
    "mlflow==2.13.2\n",
    "sagemaker-mlflow==0.1.0"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a547ea17-c14a-41b4-a0d8-26164cf24615",
   "metadata": {},
   "source": [
    "Create a Python script by moving the data processing code from the step 1 notebook to a .py file and adding experiment tracking with MLflow:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25f6f89d-c9fa-4432-bf3e-5cc21aeb2e6f",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%%writefile processing/preprocessing.py\n",
    "\n",
    "from sklearn.preprocessing import MinMaxScaler, LabelEncoder\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import argparse\n",
    "import os\n",
    "\n",
    "from time import gmtime, strftime, sleep\n",
    "import traceback\n",
    "\n",
    "import mlflow\n",
    "\n",
    "user_profile_name = os.getenv('USER')\n",
    "\n",
    "def _parse_args():\n",
    "    \n",
    "    parser = argparse.ArgumentParser()\n",
    "    # Data, model, and output directories\n",
    "    # model_dir is always passed in from SageMaker. By default this is a S3 path under the default bucket.\n",
    "    parser.add_argument('--filepath', type=str, default='/opt/ml/processing/input/')\n",
    "    parser.add_argument('--filename', type=str, default='bank-additional-full.csv')\n",
    "    parser.add_argument('--outputpath', type=str, default='/opt/ml/processing/output/')\n",
    "    \n",
    "    return parser.parse_known_args()\n",
    "\n",
    "def process_data(df_data):\n",
    "    # Indicator variable to capture when pdays takes a value of 999\n",
    "    df_data[\"no_previous_contact\"] = np.where(df_data[\"pdays\"] == 999, 1, 0)\n",
    "\n",
    "    # Indicator for individuals not actively employed\n",
    "    df_data[\"not_working\"] = np.where(\n",
    "        np.in1d(df_data[\"job\"], [\"student\", \"retired\", \"unemployed\"]), 1, 0\n",
    "    )\n",
    "\n",
    "    # remove unnecessary data\n",
    "    df_model_data = df_data.drop(\n",
    "        [\"duration\", \"emp.var.rate\", \"cons.price.idx\", \"cons.conf.idx\", \"euribor3m\", \"nr.employed\"],\n",
    "        axis=1,\n",
    "    )\n",
    "\n",
    "    bins = [18, 30, 40, 50, 60, 70, 90]\n",
    "    labels = ['18-29', '30-39', '40-49', '50-59', '60-69', '70-plus']\n",
    "\n",
    "    df_model_data['age_range'] = pd.cut(df_model_data.age, bins, labels=labels, include_lowest=True)\n",
    "    df_model_data = pd.concat([df_model_data, pd.get_dummies(df_model_data['age_range'], prefix='age', dtype=int)], axis=1)\n",
    "    df_model_data.drop('age', axis=1, inplace=True)\n",
    "    df_model_data.drop('age_range', axis=1, inplace=True)\n",
    "\n",
    "    scaled_features = ['pdays', 'previous', 'campaign']\n",
    "    df_model_data[scaled_features] = MinMaxScaler().fit_transform(df_model_data[scaled_features])\n",
    "\n",
    "    df_model_data = pd.get_dummies(df_model_data, dtype=int)  # Convert categorical variables to sets of indicators\n",
    "\n",
    "    # Replace \"y_no\" and \"y_yes\" with a single label column, and bring it to the front:\n",
    "    df_model_data = pd.concat(\n",
    "        [\n",
    "            df_model_data[\"y_yes\"].rename(target_col),\n",
    "            df_model_data.drop([\"y_no\", \"y_yes\"], axis=1),\n",
    "        ],\n",
    "        axis=1,\n",
    "    )\n",
    "    \n",
    "    return df_model_data\n",
    "\n",
    "if __name__==\"__main__\":\n",
    "    # Process arguments\n",
    "    args, _ = _parse_args()\n",
    "    \n",
    "    target_col = \"y\"\n",
    "\n",
    "    # Set the Tracking Server URI using the ARN of the Tracking Server you created\n",
    "    mlflow.set_tracking_uri(os.environ['MLFLOW_TRACKING_ARN'])\n",
    "    \n",
    "    # Enable autologging in MLflow\n",
    "    mlflow.autolog()\n",
    "\n",
    "    # Use the active run_id to log \n",
    "    with mlflow.start_run(run_id=os.environ['MLFLOW_RUN_ID']) as run:\n",
    "        # process data\n",
    "        df_model_data = process_data(pd.read_csv(os.path.join(args.filepath, args.filename), sep=\";\"))\n",
    "    \n",
    "        # Shuffle and splitting dataset\n",
    "        train_data, validation_data, test_data = np.split(\n",
    "            df_model_data.sample(frac=1, random_state=1729),\n",
    "            [int(0.7 * len(df_model_data)), int(0.9 * len(df_model_data))],\n",
    "        )\n",
    "    \n",
    "        print(f\"Data split > train:{train_data.shape} | validation:{validation_data.shape} | test:{test_data.shape}\")\n",
    "        mlflow.log_params(\n",
    "            {\n",
    "                \"train\": train_data.shape,\n",
    "                \"validate\": validation_data.shape,\n",
    "                \"test\": test_data.shape\n",
    "            }\n",
    "        )\n",
    "\n",
    "        mlflow.set_tags(\n",
    "            {\n",
    "                'mlflow.user':user_profile_name,\n",
    "                'mlflow.source.type':'JOB'\n",
    "            }\n",
    "        )\n",
    "        \n",
    "        # Save datasets locally\n",
    "        train_data.to_csv(os.path.join(args.outputpath, 'train/train.csv'), index=False, header=False)\n",
    "        validation_data.to_csv(os.path.join(args.outputpath, 'validation/validation.csv'), index=False, header=False)\n",
    "        test_data[target_col].to_csv(os.path.join(args.outputpath, 'test/test_y.csv'), index=False, header=False)\n",
    "        test_data.drop([target_col], axis=1).to_csv(os.path.join(args.outputpath, 'test/test_x.csv'), index=False, header=False)\n",
    "        \n",
    "        # Save the baseline dataset for model monitoring\n",
    "        df_model_data.drop([target_col], axis=1).to_csv(os.path.join(args.outputpath, 'baseline/baseline.csv'), index=False, header=False)\n",
    "\n",
    "        mlflow.log_artifact(local_path=os.path.join(args.outputpath, 'baseline/baseline.csv'))\n",
    "    \n",
    "    print(\"## Processing complete. Exiting.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "317a90ba-5610-43fb-a39c-6eae4e2b6691",
   "metadata": {},
   "source": [
    "The processing script contains a statement to save the whole dataset without the header and the label column as a baseline dataset. You need the data baseline later on in the model monitoring notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "70ee5a09-46fd-4d7a-9037-9b6247a1f21d",
   "metadata": {},
   "source": [
    "Set the Amazon S3 paths:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "54fa1abd-c0ae-4e55-901c-7250c227ccbb",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "train_s3_url = f\"s3://{bucket_name}/{bucket_prefix}/train\"\n",
    "validation_s3_url = f\"s3://{bucket_name}/{bucket_prefix}/validation\"\n",
    "test_s3_url = f\"s3://{bucket_name}/{bucket_prefix}/test\"\n",
    "baseline_s3_url = f\"s3://{bucket_name}/{bucket_prefix}/baseline\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ed6a2d8e-acde-48a8-9f12-a1cd0d703a41",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%store train_s3_url\n",
    "%store validation_s3_url\n",
    "%store test_s3_url\n",
    "%store baseline_s3_url"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "03762d62-0f9c-4766-8f5e-d554bfc8437c",
   "metadata": {},
   "source": [
    "Set the framework version and type and number of compute instances:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b93215af-1524-43d0-b08b-ab265936ec2a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "skprocessor_framework_version = \"1.2-1\"\n",
    "processing_instance_type = \"ml.m5.xlarge\"\n",
    "processing_instance_count = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b730b8ee-f2fe-4612-870a-edb0e0ffc276",
   "metadata": {},
   "source": [
    "#### Create a experiment run\n",
    "Create a new run in your experiment to track parameters, configuration, inputs, and outputs of the processing job."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "41e28192-db7d-4a93-8c55-de2abd6fedf6",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "run_suffix = strftime('%d-%H-%M-%S', gmtime())\n",
    "run_name = f\"container-processing-{run_suffix}\"\n",
    "\n",
    "run_id = mlflow.start_run(\n",
    "    run_name=run_name,\n",
    "    description=\"feature-engineering in the notebook 02 with a processing job\").info.run_id"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d365adc2-890e-4983-aa86-0109e1f94465",
   "metadata": {},
   "source": [
    "#### Create a processor and set inputs and outputs\n",
    "Instantiate a [`FrameworkProcessor`](https://docs.aws.amazon.com/sagemaker/latest/dg/processing-job-frameworks.html) object before starting the SageMaker processing job. You specify the instance type to use in the job, as well as how many instances for distributed processing.\n",
    "\n",
    "Note how SageMaker maps your data to the local paths on the processing container's EBS volume:\n",
    "\n",
    "![](img/data-processing.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1503fe76-cd04-4e5a-86a1-ea3cbb45f09d",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sklearn_processor = FrameworkProcessor(\n",
    "    estimator_cls=SKLearn,\n",
    "    framework_version=skprocessor_framework_version,\n",
    "    role=sm_role,\n",
    "    instance_type=processing_instance_type,\n",
    "    instance_count=processing_instance_count,\n",
    "    base_job_name='from-idea-to-prod-processing',\n",
    "    sagemaker_session=session,\n",
    "    env={\n",
    "        'MLFLOW_TRACKING_ARN': mlflow_arn,\n",
    "        'MLFLOW_RUN_ID': run_id,\n",
    "        'USER': user_profile_name\n",
    "    }\n",
    ")\n",
    "\n",
    "processing_inputs = [\n",
    "        ProcessingInput(\n",
    "            source=input_s3_url, \n",
    "            destination=\"/opt/ml/processing/input\",\n",
    "            s3_input_mode=\"File\",\n",
    "            s3_data_distribution_type=\"ShardedByS3Key\"\n",
    "        ),\n",
    "        ProcessingInput(\n",
    "            input_name=\"processor\",\n",
    "            source=dependencies_dir,\n",
    "            destination=\"/opt/ml/processing/input/code/requirements/\",\n",
    "        ),\n",
    "    ]\n",
    "\n",
    "processing_outputs = [\n",
    "        ProcessingOutput(\n",
    "            output_name=\"train_data\", \n",
    "            source=\"/opt/ml/processing/output/train\",\n",
    "            destination=train_s3_url,\n",
    "        ),\n",
    "        ProcessingOutput(\n",
    "            output_name=\"validation_data\", \n",
    "            source=\"/opt/ml/processing/output/validation\", \n",
    "            destination=validation_s3_url\n",
    "        ),\n",
    "        ProcessingOutput(\n",
    "            output_name=\"test_data\", \n",
    "            source=\"/opt/ml/processing/output/test\", \n",
    "            destination=test_s3_url\n",
    "        ),\n",
    "        ProcessingOutput(\n",
    "            output_name=\"baseline_data\", \n",
    "            source=\"/opt/ml/processing/output/baseline\", \n",
    "            destination=baseline_s3_url\n",
    "        ),\n",
    "    ]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bedef440-c13a-4d23-90af-ef747aeef8bc",
   "metadata": {},
   "source": [
    "#### Start the SageMaker processing job\n",
    "\n",
    "<img src=\"data:image/svg+xml;base64,Cjxzdmcgd2lkdGg9IjgwMCIgaGVpZ2h0PSIxMjUiIHZpZXdCb3g9IjAgMCA4MDAgMTI1IiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPgogICAgPGRlZnM+CiAgICAgICAgPGxpbmVhckdyYWRpZW50IGlkPSJmYWRlR3JhZGllbnQiIHgxPSIwIiB4Mj0iMSI+CiAgICAgICAgICAgIDxzdG9wIG9mZnNldD0iMCUiIHN0b3AtY29sb3I9IiNGMEYwRjAiLz4KICAgICAgICAgICAgPHN0b3Agb2Zmc2V0PSIxMDAlIiBzdG9wLWNvbG9yPSIjRjBGMEYwIiBzdG9wLW9wYWNpdHk9IjAiLz4KICAgICAgICA8L2xpbmVhckdyYWRpZW50PgogICAgICAgIDxtYXNrIGlkPSJmYWRlTWFzayI+CiAgICAgICAgICAgIDxyZWN0IHg9IjAiIHk9IjAiIHdpZHRoPSI3NTAiIGhlaWdodD0iMTI1IiBmaWxsPSJ3aGl0ZSIvPgogICAgICAgICAgICA8cmVjdCB4PSI3NTAiIHk9IjAiIHdpZHRoPSI1MCIgaGVpZ2h0PSIxMjUiIGZpbGw9InVybCgjZmFkZUdyYWRpZW50KSIvPgogICAgICAgIDwvbWFzaz4KICAgIDwvZGVmcz4KICAgIDxwYXRoIGQ9Ik0zLDUwIEE1MCw1MCAwIDAgMSA1MywzIEw3OTcsMyBMNzk3LDk3IEw5Nyw5NyBMNTAsMTE1IEwzLDk3IFoiIGZpbGw9IiNGMEYwRjAiIHN0cm9rZT0iI0UwRTBFMCIgc3Ryb2tlLXdpZHRoPSIxIiBtYXNrPSJ1cmwoI2ZhZGVNYXNrKSIvPgogICAgPGNpcmNsZSBjeD0iNTAiIGN5PSI1MCIgcj0iMzAiIGZpbGw9IiM1N2M0ZjgiIHN0cm9rZT0iIzU3YzRmOCIgc3Ryb2tlLXdpZHRoPSIxIi8+CiAgICA8Y2lyY2xlIGN4PSI1MCIgY3k9IjUwIiByPSIyNSIgZmlsbD0iI0YwRjBGMCIvPgogICAgPGxpbmUgeDE9IjUwIiB5MT0iNTAiIHgyPSI1MCIgeTI9IjMwIiBzdHJva2U9IiM1N2M0ZjgiIHN0cm9rZS13aWR0aD0iMyIgc3Ryb2tlLWxpbmVjYXA9InJvdW5kIi8+CiAgICA8bGluZSB4MT0iNTAiIHkxPSI1MCIgeDI9IjY1IiB5Mj0iNTAiIHN0cm9rZT0iIzU3YzRmOCIgc3Ryb2tlLXdpZHRoPSIzIiBzdHJva2UtbGluZWNhcD0icm91bmQiLz4KICAgIDx0ZXh0IHg9IjEwMCIgeT0iMzQiIGZvbnQtZmFtaWx5PSJBcmlhbCwgc2Fucy1zZXJpZiIgZm9udC1zaXplPSIxNCIgZmlsbD0iIzMzMzMzMyI+VGhlIG5leHQgY2VsbCBtYXkgdGFrZSBhIGZldyBtaW51dGVzIHRvIHJ1bi48L3RleHQ+Cjwvc3ZnPgo=\" alt=\"Time alert open medium\"/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bab839f9-6194-4467-b9d9-b207c15eb729",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "sklearn_processor.run(\n",
    "    inputs=processing_inputs,\n",
    "    outputs=processing_outputs,\n",
    "    code='processing/preprocessing.py',\n",
    "    dependencies=[f'{dependencies_dir}requirements.txt'],\n",
    "    wait=True,\n",
    "    # arguments = ['arg1', 'arg2'],\n",
    ")\n",
    "\n",
    "mlflow.set_tags(\n",
    "    {\n",
    "        'mlflow.source.name':f'https://{region}.console.aws.amazon.com/sagemaker/home?region={region}#/processing-jobs/{sklearn_processor.latest_job.name}',\n",
    "    }\n",
    ")\n",
    "\n",
    "mlflow.end_run()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d9c8e0b",
   "metadata": {},
   "source": [
    "<img src=\"data:image/svg+xml;base64,Cjxzdmcgd2lkdGg9IjgwMCIgaGVpZ2h0PSI1MCIgdmlld0JveD0iMCAwIDgwMCA1MCIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj4KICAgIDxkZWZzPgogICAgICAgIDxsaW5lYXJHcmFkaWVudCBpZD0iZmFkZUdyYWRpZW50IiB4MT0iMCIgeDI9IjEiPgogICAgICAgICAgICA8c3RvcCBvZmZzZXQ9IjAlIiBzdG9wLWNvbG9yPSIjRjBGMEYwIi8+CiAgICAgICAgICAgIDxzdG9wIG9mZnNldD0iMTAwJSIgc3RvcC1jb2xvcj0iI0YwRjBGMCIgc3RvcC1vcGFjaXR5PSIwIi8+CiAgICAgICAgPC9saW5lYXJHcmFkaWVudD4KICAgICAgICA8bWFzayBpZD0iZmFkZU1hc2siPgogICAgICAgICAgICA8cmVjdCB4PSIwIiB5PSIwIiB3aWR0aD0iNzUwIiBoZWlnaHQ9IjUwIiBmaWxsPSJ3aGl0ZSIvPgogICAgICAgICAgICA8cmVjdCB4PSI3NTAiIHk9IjAiIHdpZHRoPSI1MCIgaGVpZ2h0PSI1MCIgZmlsbD0idXJsKCNmYWRlR3JhZGllbnQpIi8+CiAgICAgICAgPC9tYXNrPgogICAgPC9kZWZzPgogICAgPHBhdGggZD0iTTI1LDUwIFEwLDUwIDAsMjUgTDUwLDMgTDk3LDI1IEw3OTcsMjUgTDc5Nyw1MCBMMjUsNTAgWiIgZmlsbD0iI0YwRjBGMCIgc3Ryb2tlPSIjRTBFMEUwIiBzdHJva2Utd2lkdGg9IjEiIG1hc2s9InVybCgjZmFkZU1hc2spIi8+Cjwvc3ZnPgo=\" alt=\"Time alert close\"/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d665ecf7-d836-40db-960e-ad8075c1ced4",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# If you set wait to False in the previous code cell, wait until the job completes\n",
    "while sm.describe_processing_job(\n",
    "        ProcessingJobName=sklearn_processor._current_job_name\n",
    "    )[\"ProcessingJobStatus\"] != \"Completed\":\n",
    "    time.sleep(10)\n",
    "    print(f\"Wait until {sklearn_processor._current_job_name} completed\")\n",
    "else:\n",
    "    print(f\"{sklearn_processor._current_job_name} completed\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "795e7281-d03b-42ba-8ef4-b0b30f9de0fd",
   "metadata": {},
   "source": [
    "To wait for job completion you can also use `boto3` [waiters](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sagemaker.html#waiters). For example:\n",
    "\n",
    "```python\n",
    "waiter = session.sagemaker_client.get_waiter('processing_job_completed_or_stopped')\n",
    "waiter.wait(ProcessingJobName=sklearn_processor._current_job_name)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ad2ceb4b-ece3-40c6-8674-ce7b8db70a4b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# list the uploaded files\n",
    "# !aws s3 ls {bucket_name}/{bucket_prefix} --recursive"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68297752",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "### Run processing script locally and remotely as a SageMaker job\n",
    "You an use [SageMaker Python SDK decorator `@remote`](https://sagemaker.readthedocs.io/en/stable/remote_function/sagemaker.remote_function.html) to run you local code in the notebook as a SageMaker processing job – called the \"Remote Function\". This is an even easier way to run your Python code at scale using SageMaker distributed processing and training.\n",
    "\n",
    "Refer to the [documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/train-remote-decorator.html) in the Amazon SageMaker developer guide.\n",
    "\n",
    "In the following section you run the data processing code as a SageMaker job using `@remote` decorator."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a8e427a9-69b3-4c85-8ba3-ab0d12faf0f2",
   "metadata": {},
   "source": [
    "#### Step 1: Develop and test you code locally\n",
    "First, you implement and test your code locally in the notebook to verify the correctness of code and environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8d9b7032-835c-4abb-8269-7a66204120e2",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Load the dataset in to DataFrame\n",
    "file_name = \"bank-additional-full.csv\"\n",
    "input_path = \"./data/bank-additional\" \n",
    "df_data = pd.read_csv(os.path.join(input_path, file_name), sep=\";\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5ab225a2-6cb0-43c7-a043-a2d0a0ba5335",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import MinMaxScaler, LabelEncoder\n",
    "\n",
    "# define a local function\n",
    "def preprocess(\n",
    "    df_data,\n",
    "    tracking_server_arn=mlflow_arn,\n",
    "    experiment_name=None,\n",
    "    run_id=None,\n",
    "):\n",
    "    import mlflow\n",
    "    from time import gmtime, strftime\n",
    "\n",
    "    try:\n",
    "        # Set the Tracking Server URI using the ARN of the Tracking Server you created\n",
    "        mlflow.set_tracking_uri(tracking_server_arn)\n",
    "        \n",
    "        # Enable autologging in MLflow\n",
    "        mlflow.autolog()\n",
    "    \n",
    "        suffix = strftime('%d-%H-%M-%S', gmtime())\n",
    "        mlflow.set_experiment(experiment_name=experiment_name if experiment_name else f\"preprocess-{suffix}\")\n",
    "        run = mlflow.start_run(run_id=run_id) if run_id else mlflow.start_run(run_name=f\"local-processing-{suffix}\", nested=True)\n",
    "    \n",
    "        target_col = \"y\"\n",
    "        \n",
    "        # Indicator variable to capture when pdays takes a value of 999\n",
    "        df_data[\"no_previous_contact\"] = np.where(df_data[\"pdays\"] == 999, 1, 0)\n",
    "    \n",
    "        # Indicator for individuals not actively employed\n",
    "        df_data[\"not_working\"] = np.where(\n",
    "            np.in1d(df_data[\"job\"], [\"student\", \"retired\", \"unemployed\"]), 1, 0\n",
    "        )\n",
    "    \n",
    "        # remove unnecessary data\n",
    "        df_model_data = df_data.drop(\n",
    "            [\"duration\", \"emp.var.rate\", \"cons.price.idx\", \"cons.conf.idx\", \"euribor3m\", \"nr.employed\"],\n",
    "            axis=1,\n",
    "        )\n",
    "    \n",
    "        bins = [18, 30, 40, 50, 60, 70, 90]\n",
    "        labels = ['18-29', '30-39', '40-49', '50-59', '60-69', '70-plus']\n",
    "    \n",
    "        df_model_data['age_range'] = pd.cut(df_model_data.age, bins, labels=labels, include_lowest=True)\n",
    "        df_model_data = pd.concat([df_model_data, pd.get_dummies(df_model_data['age_range'], prefix='age', dtype=int)], axis=1)\n",
    "        df_model_data.drop('age', axis=1, inplace=True)\n",
    "        df_model_data.drop('age_range', axis=1, inplace=True)\n",
    "    \n",
    "        scaled_features = ['pdays', 'previous', 'campaign']\n",
    "        df_model_data[scaled_features] = MinMaxScaler().fit_transform(df_model_data[scaled_features])\n",
    "    \n",
    "        df_model_data = pd.get_dummies(df_model_data, dtype=int)  # Convert categorical variables to sets of indicators\n",
    "    \n",
    "        # Replace \"y_no\" and \"y_yes\" with a single label column, and bring it to the front:\n",
    "        df_model_data = pd.concat(\n",
    "            [\n",
    "                df_model_data[\"y_yes\"].rename(target_col),\n",
    "                df_model_data.drop([\"y_no\", \"y_yes\"], axis=1),\n",
    "            ],\n",
    "            axis=1,\n",
    "        )\n",
    "    \n",
    "        # Shuffle and splitting dataset\n",
    "        train_data, validation_data, test_data = np.split(\n",
    "            df_model_data.sample(frac=1, random_state=1729),\n",
    "            [int(0.7 * len(df_model_data)), int(0.9 * len(df_model_data))],\n",
    "        )\n",
    "    \n",
    "        print(f\"Data split > train:{train_data.shape} | validation:{validation_data.shape} | test:{test_data.shape}\")\n",
    "\n",
    "        mlflow.log_params(\n",
    "            {\n",
    "                \"train\": train_data.shape,\n",
    "                \"validate\": validation_data.shape,\n",
    "                \"test\": test_data.shape\n",
    "            }\n",
    "        )\n",
    "        \n",
    "        baseline_data = df_model_data.drop([target_col], axis=1)\n",
    "        \n",
    "        print(\"## Processing complete. Exiting.\")\n",
    "        \n",
    "        return train_data, validation_data, test_data, baseline_data\n",
    "\n",
    "    except Exception as e:\n",
    "        print(f\"Exception in processing script: {e}\")\n",
    "        raise e\n",
    "    finally:\n",
    "        mlflow.end_run()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7d186016-1e10-46bc-9b2e-4ede2d86e8d5",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Call the function locally\n",
    "train_data, validation_data, test_data, baseline_data = preprocess(df_data, experiment_name=experiment_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "14fd3745-80dd-45d1-beee-f24b78717c41",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# see the processed data\n",
    "train_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eeff9ac0-798e-4e7e-8c1b-db9a86735e13",
   "metadata": {},
   "source": [
    "#### Step 2: Execute the function remotely using RemoteExecutor\n",
    "You can use [`RemoteExecutor`](https://sagemaker.readthedocs.io/en/stable/remote_function/sagemaker.remote_function.html#remoteexecutor) SageMaker Python SDK class to run the local function remotely as a SageMaker job. You can run multiple jobs in paralle using `max_parallel_jobs` parameter to control the max number of parallel jobs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5fed14c2-cea6-478c-ac96-c8310b674b9c",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sagemaker.remote_function import remote, RemoteExecutor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "72264ace-992a-42f7-9edc-6d16c063fee3",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "s3_root_uri = f\"s3://{bucket_name}/{bucket_prefix}\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9b7b1c58-b5da-4fda-b9a5-7bc7f9e05ebe",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# this code will start a SageMaker job to execute prerpocess script\n",
    "with RemoteExecutor(dependencies=f\"{dependencies_dir}requirements.txt\",\n",
    "                    s3_root_uri=s3_root_uri, instance_type=processing_instance_type) as e:\n",
    "    future = e.submit(preprocess, df_data)                 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c4b90a53-40c4-499c-84f1-985a23011aec",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "train_data, validation_data, test_data, baseline_data = future.result()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f472ed5c-2e54-4002-9fd5-818497ec8d5b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# see the processed data\n",
    "train_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "704ef0e0",
   "metadata": {},
   "source": [
    "#### Step 3: Run code with @remote decorator\n",
    "Now you can apply `@remote` to the function once the local and remote test runs successfully. \n",
    "You can also set default settings for remote functions via a [configuration file](https://docs.aws.amazon.com/sagemaker/latest/dg/train-remote-decorator-config.html). The configuration file is used when invoking a function with `@remote` decorator or `RemoteExecutor` API. You're going to use SageMaker configuration files in the notebook 3 to configure SageMaker pipelines."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "31015e89-8d1a-4ceb-9a16-6e72aa874fc6",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "@remote(dependencies=f\"{dependencies_dir}requirements.txt\",\n",
    "        s3_root_uri=s3_root_uri, instance_type=processing_instance_type)\n",
    "def preprocess(\n",
    "    df_data,\n",
    "    tracking_server_arn=mlflow_arn,\n",
    "    experiment_name=None,\n",
    "    run_id=None,\n",
    "):\n",
    "    import mlflow\n",
    "    from time import gmtime, strftime\n",
    "\n",
    "    try:\n",
    "        # Set the Tracking Server URI using the ARN of the Tracking Server you created\n",
    "        mlflow.set_tracking_uri(tracking_server_arn)\n",
    "        \n",
    "        # Enable autologging in MLflow\n",
    "        mlflow.autolog()\n",
    "    \n",
    "        suffix = strftime('%d-%H-%M-%S', gmtime())\n",
    "        mlflow.set_experiment(experiment_name=experiment_name if experiment_name else f\"preprocess-{suffix}\")\n",
    "        run = mlflow.start_run(run_id=run_id) if run_id else mlflow.start_run(run_name=f\"remote-processing-{suffix}\", nested=True)\n",
    "    \n",
    "        target_col = \"y\"\n",
    "        \n",
    "        # Indicator variable to capture when pdays takes a value of 999\n",
    "        df_data[\"no_previous_contact\"] = np.where(df_data[\"pdays\"] == 999, 1, 0)\n",
    "    \n",
    "        # Indicator for individuals not actively employed\n",
    "        df_data[\"not_working\"] = np.where(\n",
    "            np.in1d(df_data[\"job\"], [\"student\", \"retired\", \"unemployed\"]), 1, 0\n",
    "        )\n",
    "    \n",
    "        # remove unnecessary data\n",
    "        df_model_data = df_data.drop(\n",
    "            [\"duration\", \"emp.var.rate\", \"cons.price.idx\", \"cons.conf.idx\", \"euribor3m\", \"nr.employed\"],\n",
    "            axis=1,\n",
    "        )\n",
    "    \n",
    "        bins = [18, 30, 40, 50, 60, 70, 90]\n",
    "        labels = ['18-29', '30-39', '40-49', '50-59', '60-69', '70-plus']\n",
    "    \n",
    "        df_model_data['age_range'] = pd.cut(df_model_data.age, bins, labels=labels, include_lowest=True)\n",
    "        df_model_data = pd.concat([df_model_data, pd.get_dummies(df_model_data['age_range'], prefix='age', dtype=int)], axis=1)\n",
    "        df_model_data.drop('age', axis=1, inplace=True)\n",
    "        df_model_data.drop('age_range', axis=1, inplace=True)\n",
    "    \n",
    "        scaled_features = ['pdays', 'previous', 'campaign']\n",
    "        df_model_data[scaled_features] = MinMaxScaler().fit_transform(df_model_data[scaled_features])\n",
    "    \n",
    "        df_model_data = pd.get_dummies(df_model_data, dtype=int)  # Convert categorical variables to sets of indicators\n",
    "    \n",
    "        # Replace \"y_no\" and \"y_yes\" with a single label column, and bring it to the front:\n",
    "        df_model_data = pd.concat(\n",
    "            [\n",
    "                df_model_data[\"y_yes\"].rename(target_col),\n",
    "                df_model_data.drop([\"y_no\", \"y_yes\"], axis=1),\n",
    "            ],\n",
    "            axis=1,\n",
    "        )\n",
    "    \n",
    "        # Shuffle and splitting dataset\n",
    "        train_data, validation_data, test_data = np.split(\n",
    "            df_model_data.sample(frac=1, random_state=1729),\n",
    "            [int(0.7 * len(df_model_data)), int(0.9 * len(df_model_data))],\n",
    "        )\n",
    "    \n",
    "        print(f\"Data split > train:{train_data.shape} | validation:{validation_data.shape} | test:{test_data.shape}\")\n",
    "\n",
    "        mlflow.log_params(\n",
    "            {\n",
    "                \"train\": train_data.shape,\n",
    "                \"validate\": validation_data.shape,\n",
    "                \"test\": test_data.shape\n",
    "            }\n",
    "        )\n",
    "        \n",
    "        baseline_data = df_model_data.drop([target_col], axis=1)\n",
    "        \n",
    "        print(\"## Processing complete. Exiting.\")\n",
    "        \n",
    "        return train_data, validation_data, test_data, baseline_data\n",
    "\n",
    "    except Exception as e:\n",
    "        print(f\"Exception in processing script: {e}\")\n",
    "        raise e\n",
    "    finally:\n",
    "        mlflow.end_run()\n",
    "    return train_data, validation_data, test_data, baseline_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6f15213c-d877-4033-b5bf-7ab63624b308",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# This call creates and run a SageMaker job\n",
    "# This will also create a new experiment in MLflow\n",
    "train_data, validation_data, test_data, baseline_data = preprocess(df_data, experiment_name=experiment_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "efb425ed-1006-45a9-a28d-217b4ee8a77d",
   "metadata": {
    "editable": true,
    "slideshow": {
     "slide_type": ""
    },
    "tags": [
     "parameters"
    ]
   },
   "outputs": [],
   "source": [
    "# see the processed data\n",
    "train_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ff0d818-61da-43f2-aa66-4f8ee0af24db",
   "metadata": {},
   "source": [
    "For more examples of remote functions see SageMaker [example notebooks](https://docs.aws.amazon.com/sagemaker/latest/dg/train-remote-decorator-examples.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d25d03a-e988-4a73-acdd-874718d03a74",
   "metadata": {},
   "source": [
    "## Model training with SageMaker training jobs\n",
    "You can follow the same approach as with processing and now run the model training as a [SageMaker training job](https://sagemaker.readthedocs.io/en/stable/overview.html#using-estimators) with a built-in framework container in a script mode. The script mode means that you provide a script to run in one of the SageMaker pre-defined managed containers. With this approach you can focus on developing code while using standard running environments provided by SageMaker. You can also first run your code locally in a Docker container in the Studio JupyterLab space and then move to SageMaker remote jobs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9f5e3c3f-9aa5-4532-8022-95513b5002ff",
   "metadata": {},
   "outputs": [],
   "source": [
    "!mkdir -p ./training/\n",
    "!sudo rm -rf ./training-local/\n",
    "!mkdir -p ./training-local/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "99b8076b-85ed-44e4-bd08-c34d9259226e",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip show xgboost"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "459e1b53-e511-46f7-bc65-2843798a03d8",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "### Prepare the training script\n",
    "Create a `train.py` file with the training script and `requirements.txt` with the environment configuration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "75077d49-daeb-45c5-97a2-fdb15137712d",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile ./training/train.py\n",
    "\n",
    "import argparse\n",
    "import json\n",
    "import logging\n",
    "import os\n",
    "import pandas as pd\n",
    "import pickle as pkl\n",
    "\n",
    "from sagemaker_containers import entry_point\n",
    "from sagemaker_xgboost_container.data_utils import get_dmatrix\n",
    "from sagemaker_xgboost_container import distributed\n",
    "\n",
    "from sklearn.metrics import roc_auc_score\n",
    "\n",
    "import xgboost as xgb\n",
    "import mlflow\n",
    "\n",
    "from time import gmtime, strftime\n",
    "\n",
    "suffix = strftime('%d-%H-%M-%S', gmtime())\n",
    "\n",
    "user_profile_name = os.getenv('USER', 'sagemaker')\n",
    "experiment_name = os.getenv('MLFLOW_EXPERIMENT_NAME')\n",
    "region = os.getenv('REGION')\n",
    "\n",
    "mlflow.set_tracking_uri(os.getenv('MLFLOW_TRACKING_ARN'))\n",
    "mlflow.set_experiment(experiment_name=experiment_name if experiment_name else f\"train-{suffix}\")\n",
    "\n",
    "def _xgb_train(params, dtrain, dval, evals, num_boost_round, model_dir, is_master):\n",
    "    \"\"\"Run xgb train on arguments given with rabit initialized.\n",
    "\n",
    "    This is our rabit execution function.\n",
    "\n",
    "    :param args_dict: Argument dictionary used to run xgb.train().\n",
    "    :param is_master: True if current node is master host in distributed training,\n",
    "                        or is running single node training job.\n",
    "                        Note that rabit_run includes this argument.\n",
    "    \"\"\"\n",
    "    booster = xgb.train(\n",
    "        params=params,\n",
    "        dtrain=dtrain,\n",
    "        evals=evals,\n",
    "        num_boost_round=num_boost_round\n",
    "    )\n",
    "\n",
    "    val_auc = roc_auc_score(dval.get_label(), booster.predict(dval))\n",
    "    train_auc = roc_auc_score(dtrain.get_label(), booster.predict(dtrain))\n",
    "    mlflow.log_params(params)\n",
    "    mlflow.log_metrics({\"validation_auc\":val_auc, \"train_auc\":train_auc})\n",
    "    # emit training metrics - SageMaker collects them from the log stream\n",
    "    print(f\"[0]#011train-auc:{train_auc}#011validation-auc:{val_auc}\")\n",
    "    \n",
    "    if is_master:\n",
    "        model_location = model_dir + '/xgboost-model'\n",
    "        pkl.dump(booster, open(model_location, 'wb'))\n",
    "        print(\"Stored trained model at {}\".format(model_location))\n",
    "\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    parser = argparse.ArgumentParser()\n",
    "    \n",
    "    # Hyperparameters are described here.\n",
    "    parser.add_argument('--max_depth', type=int)\n",
    "    parser.add_argument('--eta', type=float)\n",
    "    parser.add_argument('--alpha', type=float)\n",
    "    parser.add_argument('--gamma', type=int)\n",
    "    parser.add_argument('--min_child_weight', type=float)\n",
    "    parser.add_argument('--subsample', type=float)\n",
    "    parser.add_argument('--colsample_bytree', type=float)\n",
    "    parser.add_argument('--verbosity', type=int)\n",
    "    parser.add_argument('--objective', type=str)\n",
    "    parser.add_argument('--num_round', type=int)\n",
    "    parser.add_argument('--early_stopping_rounds', type=int)\n",
    "    parser.add_argument('--tree_method', type=str, default=\"auto\")\n",
    "    parser.add_argument('--predictor', type=str, default=\"auto\")\n",
    "\n",
    "    # Sagemaker specific arguments. Defaults are set in the environment variables.\n",
    "    parser.add_argument('--output_data_dir', type=str, default=os.environ.get('SM_OUTPUT_DATA_DIR'))\n",
    "    parser.add_argument('--model_dir', type=str, default=os.environ.get('SM_MODEL_DIR'))\n",
    "    parser.add_argument('--train', type=str, default=os.environ.get('SM_CHANNEL_TRAIN'))\n",
    "    parser.add_argument('--validation', type=str, default=os.environ.get('SM_CHANNEL_VALIDATION'))\n",
    "    parser.add_argument('--sm_hosts', type=str, default=os.environ.get('SM_HOSTS'))\n",
    "    parser.add_argument('--sm_current_host', type=str, default=os.environ.get('SM_CURRENT_HOST'))\n",
    "    parser.add_argument('--sm_training_env', type=str, default=os.environ.get('SM_TRAINING_ENV'))\n",
    "    \n",
    "    print(\"main function\")\n",
    "    args, _ = parser.parse_known_args()\n",
    "\n",
    "    # Get SageMaker host information from runtime environment variables\n",
    "    sm_hosts = json.loads(args.sm_hosts)\n",
    "    sm_current_host = args.sm_current_host\n",
    "    dtrain = get_dmatrix(args.train, 'CSV')\n",
    "    dval = get_dmatrix(args.validation, 'CSV')\n",
    "\n",
    "    watchlist = [(dtrain, 'train'), (dval, 'validation')] if dval is not None else [(dtrain, 'train')]\n",
    "\n",
    "    # get SageMaker enviroment setup\n",
    "    sm_training_env = json.loads(args.sm_training_env)\n",
    "    \n",
    "    # enable auto logging\n",
    "    mlflow.xgboost.autolog(log_model_signatures=False, log_datasets=False)\n",
    "\n",
    "    train_hp = {\n",
    "        'max_depth': args.max_depth,\n",
    "        'eta': args.eta,\n",
    "        'gamma': args.gamma,\n",
    "        'min_child_weight': args.min_child_weight,\n",
    "        'subsample': args.subsample,\n",
    "        'verbosity': args.verbosity,\n",
    "        'objective': args.objective,\n",
    "        'tree_method': args.tree_method,\n",
    "        'predictor': args.predictor,\n",
    "    }\n",
    "\n",
    "    xgb_train_args = dict(\n",
    "        params=train_hp,\n",
    "        dtrain=dtrain,\n",
    "        dval=dval,\n",
    "        evals=watchlist,\n",
    "        num_boost_round=args.num_round,\n",
    "        model_dir=args.model_dir)\n",
    "\n",
    "    with mlflow.start_run(\n",
    "        run_name=f\"container-training-{suffix}\",\n",
    "        description=\"xgboost running in SageMaker container in script mode\"\n",
    "    ) as run:\n",
    "\n",
    "        mlflow.set_tags(\n",
    "            {\n",
    "                'mlflow.user':user_profile_name,\n",
    "                'mlflow.source.type':'JOB',\n",
    "                'mlflow.source.name': f\"https://{region}.console.aws.amazon.com/sagemaker/home?region={region}#/jobs/{sm_training_env['job_name']}\" if sm_training_env['current_host'] != 'sagemaker-local' else sm_training_env['current_host']\n",
    "            }\n",
    "        )\n",
    "    \n",
    "        if len(sm_hosts) > 1:\n",
    "            # Wait until all hosts are able to find each other\n",
    "            entry_point._wait_hostname_resolution()\n",
    "    \n",
    "            # Execute training function after initializing rabit.\n",
    "            distributed.rabit_run(\n",
    "                exec_fun=_xgb_train,\n",
    "                args=xgb_train_args,\n",
    "                include_in_training=(dtrain is not None),\n",
    "                hosts=sm_hosts,\n",
    "                current_host=sm_current_host,\n",
    "                update_rabit_args=True\n",
    "            )\n",
    "        else:\n",
    "            # If single node training, call training method directly.\n",
    "            if dtrain:\n",
    "                xgb_train_args['is_master'] = True\n",
    "                _xgb_train(**xgb_train_args)\n",
    "            else:\n",
    "                raise ValueError(\"Training channel must have data to train model.\")\n",
    "\n",
    "# Return model object\n",
    "def model_fn(model_dir):\n",
    "    \"\"\"Deserialize and return fitted model.\n",
    "\n",
    "    Note that this should have the same name as the serialized model in the _xgb_train method\n",
    "    \"\"\"\n",
    "    model_file = 'xgboost-model'\n",
    "    booster = pkl.load(open(os.path.join(model_dir, model_file), 'rb'))\n",
    "    return booster"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f0df4bc0-227e-4fb8-8df3-c583cc3a8025",
   "metadata": {},
   "source": [
    "As before for the processing job, for the training job you prepare a requirement file to be to installed the MLflow dependencies."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0254f9ce-4f60-4ac4-aa14-94c3e214f6ef",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile ./training/requirements.txt\n",
    "mlflow==2.13.2\n",
    "sagemaker-mlflow==0.1.0"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "599bf719-0736-48d4-90c9-b669ba1470ec",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "### Prepare data and hyperparameters\n",
    "Define the data input channels for the training job. Set _train_ and _validation_ channels via the SageMaker SDK [`TrainingInput`](https://sagemaker.readthedocs.io/en/stable/api/utility/inputs.html#sagemaker.inputs.TrainingInput) class:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "195881a7-985b-4b64-a370-c56e5cc638d4",
   "metadata": {},
   "outputs": [],
   "source": [
    "s3_input_train = sagemaker.inputs.TrainingInput(train_s3_url, content_type='csv')\n",
    "s3_input_validation = sagemaker.inputs.TrainingInput(validation_s3_url, content_type='csv')\n",
    "\n",
    "training_inputs = {'train': s3_input_train, 'validation': s3_input_validation}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "86c9a303-928a-4053-9002-cd618cffa6dd",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "train_instance_count = 1\n",
    "train_instance_type = \"ml.m5.xlarge\"\n",
    "\n",
    "# Define where the training job stores the model artifact\n",
    "output_s3_url = f\"s3://{bucket_name}/{bucket_prefix}/output\"\n",
    "\n",
    "%store output_s3_url"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c180ce2-8481-49ab-bcd3-06a34e36f20b",
   "metadata": {},
   "source": [
    "Prepare hyperparameters and MLflow settings in environment variables:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8af85fd6-49ed-4c56-a3fb-a50c8dc6aeb4",
   "metadata": {},
   "outputs": [],
   "source": [
    "hyperparams = {\n",
    "    'num_round': 50,\n",
    "    'max_depth': 3,\n",
    "    'eta': 0.5,\n",
    "    'alpha': 2.5,\n",
    "    'objective': 'binary:logistic',\n",
    "    'eval_metric': 'auc',\n",
    "    'subsample': 0.8,\n",
    "    'colsample_bytree': 0.8,\n",
    "    'min_child_weight': 3,\n",
    "    'early_stopping_rounds': 10,\n",
    "    'verbosity': 1\n",
    "}\n",
    "\n",
    "env_variables = {\n",
    "    'MLFLOW_TRACKING_ARN': mlflow_arn,\n",
    "    'MLFLOW_EXPERIMENT_NAME': experiment_name,\n",
    "    'USER': user_profile_name,\n",
    "    'REGION': region,\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f0821c99-fbbd-4209-a97e-8f552dd9b5dd",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "### Step 1: run training in local mode\n",
    "Running SageMaker in local mode is a convenient way to quickly iterate over your training script in the notebook to ensure it works as intended. First install the `sagemaker[local]` extras from the sagemaker SDK.\n",
    "\n",
    "<div class=\"alert alert-info\">Make sure you installed Docker in the notebook 00. You can also skip the local training and go direct to the <b>Step 2</b>.</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cf3b9a8c-cb68-41b6-8d10-e102d28dd568",
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install -q sagemaker[local]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dcc7462f-56e0-4ad4-ad57-e5703fdec7a7",
   "metadata": {},
   "source": [
    "Copy the training script to a different folder, e.g. `./training-local/` with the intention to test our script in the container locally. By doing so, you can develop your scripts fast and without need to spin up the SageMaker managed infrastructure – saving time and costs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3c8e32f3-15e9-416d-8ef2-45e5cd5f6f25",
   "metadata": {},
   "outputs": [],
   "source": [
    "%cp -rf ./training/* ./training-local"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0378bca5-26f8-4023-9601-d00795268013",
   "metadata": {},
   "source": [
    "You use the same code to run a training job as a SageMaker managed job, but set the `sagemaker_session` parameter to `LocalSession` and `instance_type` to 'local':"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "04220e95-01e3-4ceb-a54d-5e4e9133aaa1",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.xgboost.estimator import XGBoost\n",
    "from sagemaker.local import LocalSession\n",
    "\n",
    "LOCAL_SESSION = LocalSession()\n",
    "LOCAL_SESSION.config = {'local': {'local_code': True}}  # Ensure full code locality, see: https://sagemaker.readthedocs.io/en/stable/overview.html#local-mode\n",
    "\n",
    "xgb_script_mode_local = XGBoost(\n",
    "    entry_point='train.py',\n",
    "    source_dir='./training-local',\n",
    "    framework_version=\"1.7-1\",  # Note: framework_version is mandatory\n",
    "    hyperparameters=hyperparams,\n",
    "    role=sm_role,\n",
    "    instance_count=train_instance_count,\n",
    "    instance_type='local',\n",
    "    output_path=output_s3_url,\n",
    "    base_job_name=\"from-idea-to-prod-training\",\n",
    "    environment=env_variables,\n",
    "    sagemaker_session=LOCAL_SESSION,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ce167be7-7c8c-4e00-a316-d2aff4c10e52",
   "metadata": {},
   "source": [
    "<div style=\"border: 4px solid coral; text-align: center; margin: auto;\">\n",
    "First run of the training locally might take some time as the XGBoost container pulled locally.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c2cbd0a7-caf3-43f3-ac16-19104e5560fb",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">If you get an error <code>No such file or directory: 'docker'</code> in the code below, you need to run the section <b>Install Docker to enable Studio local mode</b> from the notebook 00.</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "690f37c1-97bd-474c-a257-47ac5a7733d2",
   "metadata": {},
   "outputs": [],
   "source": [
    "xgb_script_mode_local.fit(\n",
    "    training_inputs,\n",
    "    wait=True,\n",
    "    logs=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "69bdacd1-dfcf-48e6-8391-e9f1f1e53b44",
   "metadata": {},
   "source": [
    "After training is done, you can see results, metrics, and the model in the MLflow experiment. Let's construct a link to the corresponding MLflow experiment and run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c328b5b6-6177-4926-ac19-77d6471f073c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the last run in MLflow\n",
    "last_run_id = mlflow.search_runs(\n",
    "    experiment_ids=[mlflow.get_experiment_by_name(experiment_name).experiment_id], \n",
    "    max_results=1, \n",
    "    order_by=[\"attributes.start_time DESC\"]\n",
    ")['run_id'][0]\n",
    "\n",
    "# get the presigned url to open the MLflow UI\n",
    "presigned_url = sm.create_presigned_mlflow_tracking_server_url(\n",
    "    TrackingServerName=mlflow_name,\n",
    "    ExpiresInSeconds=60,\n",
    "    SessionExpirationDurationInSeconds=1800\n",
    ")['AuthorizedUrl']\n",
    "\n",
    "mlflow_run_link = f\"{presigned_url.split('/auth')[0]}/#/experiments/1/runs/{last_run_id}\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3bbd4869-b0a9-4c1b-acba-5c623eebf517",
   "metadata": {},
   "outputs": [],
   "source": [
    "# first open the MLflow UI - you can close a new opened window\n",
    "display(Javascript('window.open(\"{}\");'.format(presigned_url)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "46dde6fb-b9d6-4152-9344-83ff4c204e9a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# second open the run page in the MLflow UI\n",
    "display(Javascript('window.open(\"{}\");'.format(mlflow_run_link)))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a36532f-6634-4693-b71e-79fd6934a45c",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "### Step 2: run training as a SageMaker training job in script mode\n",
    "Use the same code to run the training script remotely on SageMaker managed infrastructure. Remove the `LocalSession` and change the `instance_type` from `local` to a desired compute instance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8296c20a-8756-49d6-82a9-8bb9efa48604",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.xgboost.estimator import XGBoost\n",
    "\n",
    "xgb_script_mode_managed = XGBoost(\n",
    "    entry_point='train.py',\n",
    "    source_dir='./training',\n",
    "    framework_version=\"1.7-1\",  \n",
    "    hyperparameters=hyperparams,\n",
    "    role=sm_role,\n",
    "    instance_count=train_instance_count,\n",
    "    instance_type=train_instance_type,\n",
    "    output_path=output_s3_url,\n",
    "    base_job_name=\"from-idea-to-prod-training\",\n",
    "    environment=env_variables,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6bb10f5c-77e4-48ed-b98e-eb17e018592d",
   "metadata": {},
   "source": [
    "<img src=\"data:image/svg+xml;base64,Cjxzdmcgd2lkdGg9IjgwMCIgaGVpZ2h0PSIxMjUiIHZpZXdCb3g9IjAgMCA4MDAgMTI1IiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPgogICAgPGRlZnM+CiAgICAgICAgPGxpbmVhckdyYWRpZW50IGlkPSJmYWRlR3JhZGllbnQiIHgxPSIwIiB4Mj0iMSI+CiAgICAgICAgICAgIDxzdG9wIG9mZnNldD0iMCUiIHN0b3AtY29sb3I9IiNGMEYwRjAiLz4KICAgICAgICAgICAgPHN0b3Agb2Zmc2V0PSIxMDAlIiBzdG9wLWNvbG9yPSIjRjBGMEYwIiBzdG9wLW9wYWNpdHk9IjAiLz4KICAgICAgICA8L2xpbmVhckdyYWRpZW50PgogICAgICAgIDxtYXNrIGlkPSJmYWRlTWFzayI+CiAgICAgICAgICAgIDxyZWN0IHg9IjAiIHk9IjAiIHdpZHRoPSI3NTAiIGhlaWdodD0iMTI1IiBmaWxsPSJ3aGl0ZSIvPgogICAgICAgICAgICA8cmVjdCB4PSI3NTAiIHk9IjAiIHdpZHRoPSI1MCIgaGVpZ2h0PSIxMjUiIGZpbGw9InVybCgjZmFkZUdyYWRpZW50KSIvPgogICAgICAgIDwvbWFzaz4KICAgIDwvZGVmcz4KICAgIDxwYXRoIGQ9Ik0zLDUwIEE1MCw1MCAwIDAgMSA1MywzIEw3OTcsMyBMNzk3LDk3IEw5Nyw5NyBMNTAsMTE1IEwzLDk3IFoiIGZpbGw9IiNGMEYwRjAiIHN0cm9rZT0iI0UwRTBFMCIgc3Ryb2tlLXdpZHRoPSIxIiBtYXNrPSJ1cmwoI2ZhZGVNYXNrKSIvPgogICAgPGNpcmNsZSBjeD0iNTAiIGN5PSI1MCIgcj0iMzAiIGZpbGw9IiM1N2M0ZjgiIHN0cm9rZT0iIzU3YzRmOCIgc3Ryb2tlLXdpZHRoPSIxIi8+CiAgICA8Y2lyY2xlIGN4PSI1MCIgY3k9IjUwIiByPSIyNSIgZmlsbD0iI0YwRjBGMCIvPgogICAgPGxpbmUgeDE9IjUwIiB5MT0iNTAiIHgyPSI1MCIgeTI9IjMwIiBzdHJva2U9IiM1N2M0ZjgiIHN0cm9rZS13aWR0aD0iMyIgc3Ryb2tlLWxpbmVjYXA9InJvdW5kIi8+CiAgICA8bGluZSB4MT0iNTAiIHkxPSI1MCIgeDI9IjY1IiB5Mj0iNTAiIHN0cm9rZT0iIzU3YzRmOCIgc3Ryb2tlLXdpZHRoPSIzIiBzdHJva2UtbGluZWNhcD0icm91bmQiLz4KICAgIDx0ZXh0IHg9IjEwMCIgeT0iMzQiIGZvbnQtZmFtaWx5PSJBcmlhbCwgc2Fucy1zZXJpZiIgZm9udC1zaXplPSIxNCIgZmlsbD0iIzMzMzMzMyI+VGhlIG5leHQgY2VsbCBtYXkgdGFrZSBhIGZldyBtaW51dGVzIHRvIHJ1bi48L3RleHQ+Cjwvc3ZnPgo=\" alt=\"Time alert open medium\"/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b01160c9-5168-4775-af31-58d73f4d5a01",
   "metadata": {},
   "outputs": [],
   "source": [
    "xgb_script_mode_managed.fit(\n",
    "    training_inputs,\n",
    "    wait=True,\n",
    "    logs=False,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ba82c386-2793-4742-9327-a1dc571c6537",
   "metadata": {},
   "source": [
    "<img src=\"data:image/svg+xml;base64,Cjxzdmcgd2lkdGg9IjgwMCIgaGVpZ2h0PSI1MCIgdmlld0JveD0iMCAwIDgwMCA1MCIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj4KICAgIDxkZWZzPgogICAgICAgIDxsaW5lYXJHcmFkaWVudCBpZD0iZmFkZUdyYWRpZW50IiB4MT0iMCIgeDI9IjEiPgogICAgICAgICAgICA8c3RvcCBvZmZzZXQ9IjAlIiBzdG9wLWNvbG9yPSIjRjBGMEYwIi8+CiAgICAgICAgICAgIDxzdG9wIG9mZnNldD0iMTAwJSIgc3RvcC1jb2xvcj0iI0YwRjBGMCIgc3RvcC1vcGFjaXR5PSIwIi8+CiAgICAgICAgPC9saW5lYXJHcmFkaWVudD4KICAgICAgICA8bWFzayBpZD0iZmFkZU1hc2siPgogICAgICAgICAgICA8cmVjdCB4PSIwIiB5PSIwIiB3aWR0aD0iNzUwIiBoZWlnaHQ9IjUwIiBmaWxsPSJ3aGl0ZSIvPgogICAgICAgICAgICA8cmVjdCB4PSI3NTAiIHk9IjAiIHdpZHRoPSI1MCIgaGVpZ2h0PSI1MCIgZmlsbD0idXJsKCNmYWRlR3JhZGllbnQpIi8+CiAgICAgICAgPC9tYXNrPgogICAgPC9kZWZzPgogICAgPHBhdGggZD0iTTI1LDUwIFEwLDUwIDAsMjUgTDUwLDMgTDk3LDI1IEw3OTcsMjUgTDc5Nyw1MCBMMjUsNTAgWiIgZmlsbD0iI0YwRjBGMCIgc3Ryb2tlPSIjRTBFMEUwIiBzdHJva2Utd2lkdGg9IjEiIG1hc2s9InVybCgjZmFkZU1hc2spIi8+Cjwvc3ZnPgo=\" alt=\"Time alert close\"/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b9289440-fa13-4bfd-a2e4-8756a37c6329",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Collect the emitted metrics from the log stream\n",
    "xgb_script_mode_managed.training_job_analytics.dataframe()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c22bca8-2bfe-4735-87ba-334e994f5ee7",
   "metadata": {},
   "source": [
    "You can see all job details in the SageMaker console:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d397d128-7273-4d3b-98fe-ed79210e0d60",
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import HTML\n",
    "\n",
    "# Show the training job link\n",
    "display(\n",
    "    HTML('<b>See the SageMaker <a target=\"top\" href=\"https://{}.console.aws.amazon.com/sagemaker/home?region={}#/jobs/{}\">training job</a></b>'.format(\n",
    "            region, region, xgb_script_mode_managed.latest_training_job.name))\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cee49eb6-f42c-47d2-9738-01da2f2c7669",
   "metadata": {},
   "source": [
    "Click on the link ^^^ above ^^^ to open the SageMaker console with the training job details."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fc870dab-1d90-4855-b1c7-4c58cff64a43",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "### Optional: train with SageMaker built-in algorithms\n",
    "Instead of developing your own script, you can use one of the SageMaker [build-in algorithms](https://docs.aws.amazon.com/sagemaker/latest/dg/algos.html). In this section you're going to use the [XGBoost](https://docs.aws.amazon.com/sagemaker/latest/dg/xgboost.html) algorithm. You don't need to provide your own training script. You only need to instantiate an [Estimator](https://sagemaker.readthedocs.io/en/stable/api/training/estimators.html) object, set algorithm's [hyperparameters](https://docs.aws.amazon.com/sagemaker/latest/dg/xgboost_hyperparameters.html), and call `fit()` method of the estimator."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dfbeec6d-419d-4d88-a3f6-194440dd0124",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# get training container uri\n",
    "training_image = sagemaker.image_uris.retrieve(\"xgboost\", region=region, version=\"1.7-1\")\n",
    "\n",
    "print(training_image)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "db1a89cc-c841-4e5d-99b3-d4fe5b6435cc",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Instantiate an XGBoost estimator object\n",
    "estimator = sagemaker.estimator.Estimator(\n",
    "    image_uri=training_image,  # XGBoost algorithm container\n",
    "    instance_type=train_instance_type,  # type of training instance\n",
    "    instance_count=train_instance_count,  # number of instances to be used\n",
    "    role=sm_role,  # IAM execution role to be used\n",
    "    max_run=20 * 60,  # Maximum allowed active runtime\n",
    "    # use_spot_instances=True,  # Use spot instances to reduce cost\n",
    "    # max_wait=30 * 60,  # Maximum clock time (including spot delays)\n",
    "    output_path=output_s3_url, # S3 location for saving the training result\n",
    "    sagemaker_session=session, # Session object which manages interactions with SageMaker API and AWS services\n",
    "    base_job_name=\"from-idea-to-prod-training\", # Prefix for training job name\n",
    ")\n",
    "\n",
    "# define its hyperparameters\n",
    "estimator.set_hyperparameters(\n",
    "    num_round=50, # the number of rounds to run the training\n",
    "    max_depth=3, # maximum depth of a tree\n",
    "    eta=0.5, # step size shrinkage used in updates to prevent overfitting\n",
    "    alpha=2.5, # L1 regularization term on weights\n",
    "    objective=\"binary:logistic\",\n",
    "    eval_metric=\"auc\", # evaluation metrics for validation data\n",
    "    subsample=0.8, # subsample ratio of the training instance\n",
    "    colsample_bytree=0.8, # subsample ratio of columns when constructing each tree\n",
    "    min_child_weight=3, # minimum sum of instance weight (hessian) needed in a child\n",
    "    early_stopping_rounds=10, # the model trains until the validation score stops improving\n",
    "    verbosity=1, # verbosity of printing messages\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c8ee39d9-70d8-4f97-b749-9cc5b09e70c8",
   "metadata": {},
   "outputs": [],
   "source": [
    "estimator.hyperparameters()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5f7886c8-7bdd-4659-b045-8fac4b2755e1",
   "metadata": {},
   "outputs": [],
   "source": [
    "# helper function to load XGBoost model into xgboost.Booster\n",
    "def load_model(model_data_s3_uri):\n",
    "    import xgboost as xgb\n",
    "    import tarfile\n",
    "    import pickle as pkl\n",
    "\n",
    "    model_file = \"./xgboost-model.tar.gz\"\n",
    "    bucket, key = model_data_s3_uri.replace(\"s3://\", \"\").split(\"/\", 1)\n",
    "    boto3.client(\"s3\").download_file(bucket, key, model_file)\n",
    "    \n",
    "    with tarfile.open(model_file, \"r:gz\") as t:\n",
    "        t.extractall(path=\".\")\n",
    "    \n",
    "    # Load model\n",
    "    model = xgb.Booster()\n",
    "    model.load_model(\"xgboost-model\")\n",
    "\n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3022218-4f87-4e59-8f58-9bab2a44bb96",
   "metadata": {},
   "source": [
    "Run the training:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ca5a53b",
   "metadata": {},
   "source": [
    "<img src=\"data:image/svg+xml;base64,Cjxzdmcgd2lkdGg9IjgwMCIgaGVpZ2h0PSIxMjUiIHZpZXdCb3g9IjAgMCA4MDAgMTI1IiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPgogICAgPGRlZnM+CiAgICAgICAgPGxpbmVhckdyYWRpZW50IGlkPSJmYWRlR3JhZGllbnQiIHgxPSIwIiB4Mj0iMSI+CiAgICAgICAgICAgIDxzdG9wIG9mZnNldD0iMCUiIHN0b3AtY29sb3I9IiNGMEYwRjAiLz4KICAgICAgICAgICAgPHN0b3Agb2Zmc2V0PSIxMDAlIiBzdG9wLWNvbG9yPSIjRjBGMEYwIiBzdG9wLW9wYWNpdHk9IjAiLz4KICAgICAgICA8L2xpbmVhckdyYWRpZW50PgogICAgICAgIDxtYXNrIGlkPSJmYWRlTWFzayI+CiAgICAgICAgICAgIDxyZWN0IHg9IjAiIHk9IjAiIHdpZHRoPSI3NTAiIGhlaWdodD0iMTI1IiBmaWxsPSJ3aGl0ZSIvPgogICAgICAgICAgICA8cmVjdCB4PSI3NTAiIHk9IjAiIHdpZHRoPSI1MCIgaGVpZ2h0PSIxMjUiIGZpbGw9InVybCgjZmFkZUdyYWRpZW50KSIvPgogICAgICAgIDwvbWFzaz4KICAgIDwvZGVmcz4KICAgIDxwYXRoIGQ9Ik0zLDUwIEE1MCw1MCAwIDAgMSA1MywzIEw3OTcsMyBMNzk3LDk3IEw5Nyw5NyBMNTAsMTE1IEwzLDk3IFoiIGZpbGw9IiNGMEYwRjAiIHN0cm9rZT0iI0UwRTBFMCIgc3Ryb2tlLXdpZHRoPSIxIiBtYXNrPSJ1cmwoI2ZhZGVNYXNrKSIvPgogICAgPGNpcmNsZSBjeD0iNTAiIGN5PSI1MCIgcj0iMzAiIGZpbGw9IiM1N2M0ZjgiIHN0cm9rZT0iIzU3YzRmOCIgc3Ryb2tlLXdpZHRoPSIxIi8+CiAgICA8Y2lyY2xlIGN4PSI1MCIgY3k9IjUwIiByPSIyNSIgZmlsbD0iI0YwRjBGMCIvPgogICAgPGxpbmUgeDE9IjUwIiB5MT0iNTAiIHgyPSI1MCIgeTI9IjMwIiBzdHJva2U9IiM1N2M0ZjgiIHN0cm9rZS13aWR0aD0iMyIgc3Ryb2tlLWxpbmVjYXA9InJvdW5kIi8+CiAgICA8bGluZSB4MT0iNTAiIHkxPSI1MCIgeDI9IjY1IiB5Mj0iNTAiIHN0cm9rZT0iIzU3YzRmOCIgc3Ryb2tlLXdpZHRoPSIzIiBzdHJva2UtbGluZWNhcD0icm91bmQiLz4KICAgIDx0ZXh0IHg9IjEwMCIgeT0iMzQiIGZvbnQtZmFtaWx5PSJBcmlhbCwgc2Fucy1zZXJpZiIgZm9udC1zaXplPSIxNCIgZmlsbD0iIzMzMzMzMyI+VGhlIG5leHQgY2VsbCBtYXkgdGFrZSBhIGZldyBtaW51dGVzIHRvIHJ1bi48L3RleHQ+Cjwvc3ZnPgo=\" alt=\"Time alert open medium\"/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "287da7a9-c0ca-479e-8677-24176d91f95f",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "mlflow.set_experiment(experiment_name=experiment_name)\n",
    "with mlflow.start_run(\n",
    "    run_name=f\"container-training-{strftime('%d-%H-%M-%S', gmtime())}\",\n",
    "    description=\"training in the notebook 02 with a training job\") as run:\n",
    "    mlflow.log_params(estimator.hyperparameters())\n",
    "        \n",
    "    estimator.fit(\n",
    "        training_inputs,\n",
    "        wait=True,\n",
    "        logs=False,\n",
    "    ) \n",
    "    \n",
    "    mlflow.set_tags(\n",
    "        {\n",
    "            'mlflow.user':user_profile_name,\n",
    "            'mlflow.source.name':f'https://{region}.console.aws.amazon.com/sagemaker/home?region={region}#/jobs/{estimator.latest_training_job.name}',\n",
    "            'mlflow.source.type':'JOB'\n",
    "        }\n",
    "    )\n",
    "    mlflow.log_param(\"training job name\", estimator.latest_training_job.name)\n",
    "    mlflow.log_metrics({i['metric_name'].replace(':', '_'):i['value'] for i in estimator.training_job_analytics.dataframe().iloc})\n",
    "    mlflow.xgboost.log_model(load_model(estimator.model_data), artifact_path=\"model\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "acf59a4c",
   "metadata": {},
   "source": [
    "<img src=\"data:image/svg+xml;base64,Cjxzdmcgd2lkdGg9IjgwMCIgaGVpZ2h0PSI1MCIgdmlld0JveD0iMCAwIDgwMCA1MCIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj4KICAgIDxkZWZzPgogICAgICAgIDxsaW5lYXJHcmFkaWVudCBpZD0iZmFkZUdyYWRpZW50IiB4MT0iMCIgeDI9IjEiPgogICAgICAgICAgICA8c3RvcCBvZmZzZXQ9IjAlIiBzdG9wLWNvbG9yPSIjRjBGMEYwIi8+CiAgICAgICAgICAgIDxzdG9wIG9mZnNldD0iMTAwJSIgc3RvcC1jb2xvcj0iI0YwRjBGMCIgc3RvcC1vcGFjaXR5PSIwIi8+CiAgICAgICAgPC9saW5lYXJHcmFkaWVudD4KICAgICAgICA8bWFzayBpZD0iZmFkZU1hc2siPgogICAgICAgICAgICA8cmVjdCB4PSIwIiB5PSIwIiB3aWR0aD0iNzUwIiBoZWlnaHQ9IjUwIiBmaWxsPSJ3aGl0ZSIvPgogICAgICAgICAgICA8cmVjdCB4PSI3NTAiIHk9IjAiIHdpZHRoPSI1MCIgaGVpZ2h0PSI1MCIgZmlsbD0idXJsKCNmYWRlR3JhZGllbnQpIi8+CiAgICAgICAgPC9tYXNrPgogICAgPC9kZWZzPgogICAgPHBhdGggZD0iTTI1LDUwIFEwLDUwIDAsMjUgTDUwLDMgTDk3LDI1IEw3OTcsMjUgTDc5Nyw1MCBMMjUsNTAgWiIgZmlsbD0iI0YwRjBGMCIgc3Ryb2tlPSIjRTBFMEUwIiBzdHJva2Utd2lkdGg9IjEiIG1hc2s9InVybCgjZmFkZU1hc2spIi8+Cjwvc3ZnPgo=\" alt=\"Time alert close\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f96001de-6461-4e1d-b4a1-fa6208a2d0db",
   "metadata": {},
   "source": [
    "See now the details of the training job by clicking on the link constructed by the cell below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a2209bbe-0a9d-4306-863b-df16c3cafa97",
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import HTML\n",
    "\n",
    "# Show the training job link\n",
    "display(\n",
    "    HTML('<b>See the SageMaker <a target=\"top\" href=\"https://{}.console.aws.amazon.com/sagemaker/home?region={}#/jobs/{}\">training job</a></b>'.format(\n",
    "            region, region, estimator.latest_training_job.name))\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5283ce86-6630-4b0a-a341-48cda396d18f",
   "metadata": {},
   "source": [
    "#### Output model performance from the estimator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "052e9640-3a0b-4a30-a897-ecd2ee102ca4",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "if estimator._current_job_name:\n",
    "    training_job_name = estimator._current_job_name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e83a2509-8e2c-4e1c-874f-215745be46e2",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%store training_job_name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a69d7a0c-8cbe-4fd1-af44-9677107b5229",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "metrics = None\n",
    "while not metrics:\n",
    "    metrics = sm.describe_training_job(\n",
    "        TrainingJobName=training_job_name\n",
    "        ).get(\"FinalMetricDataList\")\n",
    "\n",
    "    if not metrics:\n",
    "        print(f\"Training job {training_job_name} hasn't finished yet!\")\n",
    "        time.sleep(10)\n",
    "    \n",
    "train_auc = float([m['Value'] for m in metrics if m['MetricName'] == 'train:auc'][0])\n",
    "validate_auc = float([m['Value'] for m in metrics if m['MetricName'] == 'validation:auc'][0])\n",
    "\n",
    "print(f\"Train-auc:{train_auc:.4f}, Validate-auc:{validate_auc:.4f}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "05a41f7a-4f7e-413b-9f5f-a7557bf4060a",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Print the S3 path to the model artifact:\n",
    "estimator.model_data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c6d7df7-fec8-474b-a111-1f5ced4d53c6",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "### Register the model in the MLflow model registry\n",
    "Now register the trained model in the MLflow model registry. The model is also automatically registered in the SageMaker model registry.\n",
    "Note that the following script registers the model trained by the latest experiment run. Depending on which training option or options you choosen, the model could be from a local training run, a training job in script mode, or from a built-in SageMaker algorithm."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a1d1343b-d6b3-4012-96a0-6636c81ef02e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the last run in MLflow\n",
    "last_run_id = mlflow.search_runs(\n",
    "    experiment_ids=[mlflow.get_experiment_by_name(experiment_name).experiment_id], \n",
    "    max_results=1, \n",
    "    order_by=[\"attributes.start_time DESC\"]\n",
    ")['run_id'][0]\n",
    "\n",
    "# construct the model URI\n",
    "model_uri = f\"runs:/{last_run_id}/model\"\n",
    "\n",
    "# register the model\n",
    "registered_model_version = mlflow.register_model(model_uri, registered_model_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "00f495df-8985-4d11-ae67-1fa82382afe0",
   "metadata": {},
   "source": [
    "You can also see the model metrics from the MLflow registered model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "85df1eba-5cdb-4875-a909-5a9c33a122c7",
   "metadata": {},
   "outputs": [],
   "source": [
    "mlflow.get_run(registered_model_version.run_id).data.metrics"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4525ff7c-527d-4cd3-8708-83d516e47a02",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "### Optional: Reduce training job startup time with warm pools\n",
    "\n",
    "<div class=\"alert alert-info\">This section is optional – you don't need it for the further course of the workshop. <b>Do not run this section in an AWS-provisioned workshop account</b>.</div>\n",
    "\n",
    "Instead of using each time a new ephemeral computation cluster to train your models, you can keep your model training hardware instances warm after every job for a specified period. Refer to [Reduce ML Model Training Job startup time by up to 8x using SageMaker Training Managed Warm Pools](https://aws.amazon.com/about-aws/whats-new/2022/09/reduce-ml-model-training-job-startup-time-8x-sagemaker-training-managed-warm-pools/) for more details. If you opt to use warm pools, you are billed for the instances and EBS volumes for the duration of the keep-alive period. \n",
    "Refer to [ Train Using SageMaker Managed Warm Pools](https://docs.aws.amazon.com/sagemaker/latest/dg/train-warm-pools.html) in the Amazon SageMaker Developer Guide for details on training API."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a7af5386",
   "metadata": {},
   "source": [
    "<div style=\"border: 4px solid coral; text-align: center; margin: auto;\">\n",
    "    <p style=\" text-align: center; margin: auto;\">To use warm pool feature you must have a corresponding warm pool quota for a required instance type set to value greater than 0.\n",
    "    <br>\n",
    "    <br>\n",
    "    Do not run this section in an AWS provisioned workshop account as the warm pool quota is set to 0.\n",
    "    The following section checks the quota value for the training instance type.\n",
    "    </p>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "909030b3-4db3-4229-a1ce-5ee453e4b88e",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def check_quota(quota_code, min_v):\n",
    "    r = quotas_client.get_service_quota(\n",
    "        ServiceCode=\"sagemaker\",\n",
    "        QuotaCode=quota_code,\n",
    "    )\n",
    "    \n",
    "    q = r[\"Quota\"][\"Value\"]\n",
    "    n = r[\"Quota\"][\"QuotaName\"]\n",
    "\n",
    "    if q < min_v:\n",
    "        print (\n",
    "            f\"WARNING: Your quota {q} for {n} is less than required value of {min_v}\"\n",
    "        )\n",
    "    else:\n",
    "        print(\n",
    "            f\"SUCCESS: Your quota {q} for {n} is equal or more than required value of {min_v}\"\n",
    "        )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "60632a28-8cc2-45ca-8e21-dbb8ea38f4b7",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "quotas_client = boto3.client(\"service-quotas\")\n",
    "                      \n",
    "quotas = {\n",
    "    \"ml.m5.large\": [\"L-2DD73636\", 1],\n",
    "    \"ml.m5.xlarge\": [\"L-0BEF44E8\", 1],\n",
    "    \"ml.m5.2xlarge\": [\"L-1686EE8B\", 1],\n",
    "}\n",
    "     \n",
    "check_quota(quotas[train_instance_type][0], quotas[train_instance_type][1])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd3f1e66",
   "metadata": {},
   "source": [
    "#### Train with SageMaker warm pools\n",
    "Let's use this feature and run XGBoost training using warm pools.\n",
    "Notice the matching attributes of a training job to re-use the provisioned infrastructure from a previous job: [Matching criteria](https://docs.aws.amazon.com/sagemaker/latest/dg/train-warm-pools.html#train-warm-pools-matching-criteria)\n",
    "\n",
    "To create a warm pool you need to set `KeepAlivePeriodInSeconds` parameter in `Estimator` configuration to value greater than 0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0e13412d",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Instantiate an XGBoost estimator object\n",
    "warm_pool_estimator = sagemaker.estimator.Estimator(\n",
    "    image_uri=training_image,  # XGBoost algorithm container\n",
    "    instance_type=train_instance_type,  # type of training instance\n",
    "    instance_count=train_instance_count,  # number of instances to be used\n",
    "    role=sm_role,  # IAM execution role to be used\n",
    "    max_run=20 * 60,  # Maximum allowed active runtime\n",
    "    # use_spot_instances=True,  # Use spot instances to reduce cost\n",
    "    # max_wait=30 * 60,  # Maximum clock time (including spot delays)\n",
    "    output_path=output_s3_url, # S3 location for saving the training result\n",
    "    sagemaker_session=session, # Session object which manages interactions with SageMaker API and AWS services\n",
    "    base_job_name=\"from-idea-to-prod-training\", # Prefix for training job name\n",
    "    keep_alive_period_in_seconds=1800, # use the warm pool feature\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c0a7a049",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "training_inputs = {'train': s3_input_train, 'validation': s3_input_validation}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5688fd05",
   "metadata": {},
   "source": [
    "Run a training job by calling `estimator.fit()` several consequent times with different hyperparameters. The initial training job \"cold-starts\" as SageMaker provisions required compute infrastructure for it. When this job completes, the infrastructure kept alive for the period `KeepAlivePeriodInSeconds`. The warm pool stays `Available` until it either identifies a matching training job for reuse or it exceeds the specified `KeepAlivePeriodInSeconds` and is terminated."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1035d322-d47d-4357-bbb8-ac85f8558b48",
   "metadata": {},
   "source": [
    "Follow job executions in the SageMaker console by clicking on the link constructed by the following cell:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c4df95f5-6a58-4871-83b1-678d2e427787",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Show the training jobs link\n",
    "display(\n",
    "    HTML('<b>See SageMaker <a target=\"top\" href=\"https://{}.console.aws.amazon.com/sagemaker/home?region={}#/jobs/\">training jobs</a></b>'.format(\n",
    "            region, region))\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "628ca868",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Start a new experiment to log execution times for each estimator fit\n",
    "suffix = strftime('%d-%H-%M-%S', gmtime())\n",
    "mlflow.set_experiment(experiment_name=f\"from-idea-to-prod-warm-pools-{suffix}\")\n",
    "\n",
    "with mlflow.start_run(\n",
    "    run_name=f\"container-warm-pools-{suffix}\",\n",
    "    description=\"warm pools experiment in the notebook 02\") as parent_run:\n",
    "    # run the training job five times\n",
    "    for i, d in enumerate([2, 3, 5, 10, 20]):\n",
    "        print(f\"Fit estimator with max_depth={d}\")\n",
    "    \n",
    "        warm_pool_estimator.set_hyperparameters(\n",
    "            num_round=50, # the number of rounds to run the training\n",
    "            max_depth=d, # maximum depth of a tree\n",
    "            objective=\"binary:logistic\",\n",
    "            eval_metric=\"auc\", # evaluation metrics for validation data\n",
    "            early_stopping_rounds=10, # the model trains until the validation score stops improving\n",
    "        )\n",
    "\n",
    "        with mlflow.start_run(\n",
    "            run_name=f\"max_depth={d}\",\n",
    "            description=f\"Fit estimator with max_depth={d}\",\n",
    "            nested=True) as child_run:\n",
    "            mlflow.log_params(warm_pool_estimator.hyperparameters())\n",
    "            \n",
    "            warm_pool_estimator.fit(\n",
    "                training_inputs,\n",
    "                wait=True,\n",
    "                logs=False,\n",
    "            )\n",
    "            \n",
    "            mlflow.log_metrics({i['metric_name'].replace(':', '_'):i['value'] for i in warm_pool_estimator.training_job_analytics.dataframe().iloc})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a248adc4-3d60-4d5c-9b25-923b51473de0",
   "metadata": {},
   "source": [
    "You can validate that a warm pool used for this training job by going to the [SageMaker training job console](https://console.aws.amazon.com/sagemaker/home?#/jobs) and inspect the training job list:\n",
    "\n",
    "![](img/warm-pools-training-jobs.png)\n",
    "\n",
    "The first training job should take about several minutes, but all subsequent jobs reuse the same compute instance and completed in several seconds. You can also see the warm pool status and time left.\n",
    "\n",
    "You can also open an MLflow experiment list and select an `from-idea-to-prod-warm-pools-<timestamp>` experiment. The logged run duration shows that the first training took about 2 min and all consequent runs less than 40 sec:\n",
    "\n",
    "![](img/warm-pools-mlflow.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1aa43bd9-7cab-48da-b128-635b90c49971",
   "metadata": {},
   "source": [
    "## Deploy and test the model\n",
    "All training jobs saved a model package in the specified location on Amazon S3.\n",
    "\n",
    "You can deploy the model as a [real-time endpoint](https://docs.aws.amazon.com/sagemaker/latest/dg/realtime-endpoints.html), which is just one [function call](https://sagemaker.readthedocs.io/en/stable/api/training/estimators.html#sagemaker.estimator.Estimator.deploy), or create a [batch transform](https://docs.aws.amazon.com/sagemaker/latest/dg/batch-transform.html) to predict a label for a large dataset. You can also deploy model to an endpoint from the MLflow model registry by using the [ModelBuilder](https://docs.aws.amazon.com/sagemaker/latest/dg/how-it-works-modelbuilder-creation.html) class."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4358692a-a75a-4d57-9236-5932fc45edc4",
   "metadata": {},
   "source": [
    "### Prepare the test data\n",
    "Download the prepared test dataset from S3 to the notebook:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4aee60dd-7238-48cb-8ae7-9765f517c889",
   "metadata": {},
   "outputs": [],
   "source": [
    "!aws s3 cp $test_s3_url/test_x.csv tmp/test_x.csv\n",
    "!aws s3 cp $test_s3_url/test_y.csv tmp/test_y.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9b27d32-470e-4eed-9eba-3681938f46cb",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "### Local inference in the notebook\n",
    "First run inference locally using a registered model from the MLflow model registry."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7dc5608f-46a2-4e40-8b2e-7ce20502c981",
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the model from the MLflow model registry\n",
    "model_uri = registered_model_version.source\n",
    "model = mlflow.xgboost.load_model(model_uri)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "352679e2-d7f9-47ec-bac3-d0c83e45b1f8",
   "metadata": {},
   "outputs": [],
   "source": [
    "import xgboost as xgb\n",
    "from numpy import loadtxt\n",
    "\n",
    "test_x = loadtxt(\"tmp/test_x.csv\", delimiter=\",\")\n",
    "test_y = loadtxt(\"tmp/test_y.csv\", delimiter=\",\")\n",
    "\n",
    "dtest = xgb.DMatrix(test_x)\n",
    "\n",
    "predictions = np.array(model.predict(dtest), dtype=float).squeeze()\n",
    "predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0de499c2-03a7-498e-ba5b-a97e6e2508bd",
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.crosstab(\n",
    "    index=test_y,\n",
    "    columns=np.round(predictions), \n",
    "    rownames=['actuals'], \n",
    "    colnames=['predictions']\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d2b02426-0b29-4b42-a182-7e62ba951767",
   "metadata": {},
   "source": [
    "### Real-time inference with a SageMaker endpoint\n",
    "\n",
    "<div class=\"alert alert-info\"><code>ModelBuilder</code> doesn work in SageMaker Distribution Image 2.0 and 2.1 because of version incompatibility of <code>cloudpickle</code>. To run this section you can use SageMaker Distribution Image 1.11</div>\n",
    "\n",
    "In this section you create a real-time [SageMaker endpoint](https://docs.aws.amazon.com/sagemaker/latest/dg/realtime-endpoints.html) using the MLflow model and the [ModelBuilder](https://docs.aws.amazon.com/sagemaker/latest/dg/how-it-works-modelbuilder-creation.html) Python SDK class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "f5492e6d-fda1-4a32-bc1e-b91cf564eedc",
   "metadata": {},
   "outputs": [],
   "source": [
    "smd_version = sm.describe_space(DomainId=domain_id, SpaceName=space_name)['SpaceSettings']['JupyterLabAppSettings']['DefaultResourceSpec']['SageMakerImageVersionAlias']\n",
    "if smd_version.split('.')[0] != '1':\n",
    "    print(\"\\033[91mModelBuilder class doesn't work in SageMaker Distribution Image 2.x. Use SMD 1.11 to run this section\\033[0m\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "483bdcfd-4e6a-420d-921e-61cad7048573",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "#### Build the deployable model\n",
    "Use the latest registered model stored in `registered_model_version` to deploy the model to a SageMaker real-time inference endpoint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a32e4dd0-15ee-41de-bbca-a906206cde8f",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sagemaker.serve import (\n",
    "    CustomPayloadTranslator,\n",
    "    InferenceSpec,\n",
    "    ModelBuilder,\n",
    "    SchemaBuilder,\n",
    "    Mode\n",
    ")\n",
    "from pathlib import Path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b0d5a344-430a-48d0-81ad-7f5b7184cf00",
   "metadata": {},
   "outputs": [],
   "source": [
    "# all model artifacts are stored in MLflow artifact directory on S3\n",
    "!aws s3 ls {registered_model_version.source} --recursive"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6b054b13-ac04-41c5-81e9-90340e0bb808",
   "metadata": {},
   "outputs": [],
   "source": [
    "# download the model artifact to the notebook's local file system\n",
    "artifact_path = Path(\"artifact-dir\")\n",
    "artifact_path.mkdir(exist_ok=True)\n",
    "artifact_dir = artifact_path.as_posix()\n",
    "mlflow.tracking.artifact_utils._download_artifact_from_uri(\n",
    "    registered_model_version.source,\n",
    "    output_path=artifact_dir,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "018136de-c48e-4dcc-822f-b0dde4581a20",
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a custom inference spec class\n",
    "class MyXGBoostModel(InferenceSpec):\n",
    "    def invoke(self, input_object: object, model: object):\n",
    "        import xgboost as xgb\n",
    "        input_converted = xgb.DMatrix(input_object)\n",
    "        output = model.predict(input_converted)\n",
    "        return output\n",
    "\n",
    "    def load(self, model_dir: str):\n",
    "        model = mlflow.xgboost.load_model(model_dir)\n",
    "        return model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "55c62c9d-d70c-482d-95b8-b36cd2e25077",
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a model builder\n",
    "model_builder = ModelBuilder(\n",
    "    mode=Mode.SAGEMAKER_ENDPOINT,\n",
    "    model_path=\"./artifact-dir/model\",\n",
    "    inference_spec=MyXGBoostModel(),\n",
    "    schema_builder=SchemaBuilder(sample_input=test_x, sample_output=test_y),  \n",
    "    role_arn=sm_role,\n",
    "    image_uri=sagemaker.image_uris.retrieve(\"xgboost\", region, version=\"1.7-1\"),\n",
    "    dependencies={\n",
    "        \"requirements\": \"./artifact-dir/model/requirements.txt\"\n",
    "    },\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c07f65d-506e-4314-b7aa-6fa65024cc9a",
   "metadata": {},
   "source": [
    "The last step is to call `ModelBuilder.build()`. This call creates inference code (as `inference.py`) in your working directory with the code necessary to create your schema, run serialization and deserialization of inputs and outputs, and run other user-specified custom logic."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c543b396-93f1-4dfe-9a3d-5fe2f1c16108",
   "metadata": {},
   "outputs": [],
   "source": [
    "# create a deployable model\n",
    "deployable_model = model_builder.build()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fd90476a-abe4-4402-a7d2-20daac4b40d0",
   "metadata": {},
   "source": [
    "For more examples how to use `ModelBuilder`, for example for a custom model serving container, refer to [sample notebooks](https://github.com/aws-samples/sagemaker-hosting/tree/main/SageMaker-Model-Builder)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17c01d22-f9e2-451b-85cd-1bfa33ed5003",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "#### Deploy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "06b2389a-bfc0-4ef3-b2b9-a139a8b9e994",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Deploy the model as a real-time endpoint\n",
    "endpoint_name = f\"from-idea-to-prod-endpoint-{strftime('%d-%H-%M-%S', gmtime())}\"\n",
    "\n",
    "predictor = deployable_model.deploy(\n",
    "    initial_instance_count=1, \n",
    "    instance_type=\"ml.m5.xlarge\",\n",
    "    endpoint_name=endpoint_name,\n",
    "    wait=False,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b97c0686-b9a5-4f21-9cb9-453b73dc14f6",
   "metadata": {},
   "source": [
    "<img src=\"data:image/svg+xml;base64,Cjxzdmcgd2lkdGg9IjgwMCIgaGVpZ2h0PSIxMjUiIHZpZXdCb3g9IjAgMCA4MDAgMTI1IiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPgogICAgPGRlZnM+CiAgICAgICAgPGxpbmVhckdyYWRpZW50IGlkPSJmYWRlR3JhZGllbnQiIHgxPSIwIiB4Mj0iMSI+CiAgICAgICAgICAgIDxzdG9wIG9mZnNldD0iMCUiIHN0b3AtY29sb3I9IiNGMEYwRjAiLz4KICAgICAgICAgICAgPHN0b3Agb2Zmc2V0PSIxMDAlIiBzdG9wLWNvbG9yPSIjRjBGMEYwIiBzdG9wLW9wYWNpdHk9IjAiLz4KICAgICAgICA8L2xpbmVhckdyYWRpZW50PgogICAgICAgIDxtYXNrIGlkPSJmYWRlTWFzayI+CiAgICAgICAgICAgIDxyZWN0IHg9IjAiIHk9IjAiIHdpZHRoPSI3NTAiIGhlaWdodD0iMTI1IiBmaWxsPSJ3aGl0ZSIvPgogICAgICAgICAgICA8cmVjdCB4PSI3NTAiIHk9IjAiIHdpZHRoPSI1MCIgaGVpZ2h0PSIxMjUiIGZpbGw9InVybCgjZmFkZUdyYWRpZW50KSIvPgogICAgICAgIDwvbWFzaz4KICAgIDwvZGVmcz4KICAgIDxwYXRoIGQ9Ik0zLDUwIEE1MCw1MCAwIDAgMSA1MywzIEw3OTcsMyBMNzk3LDk3IEw5Nyw5NyBMNTAsMTE1IEwzLDk3IFoiIGZpbGw9IiNGMEYwRjAiIHN0cm9rZT0iI0UwRTBFMCIgc3Ryb2tlLXdpZHRoPSIxIiBtYXNrPSJ1cmwoI2ZhZGVNYXNrKSIvPgogICAgPGNpcmNsZSBjeD0iNTAiIGN5PSI1MCIgcj0iMzAiIGZpbGw9IiM1N2M0ZjgiIHN0cm9rZT0iIzU3YzRmOCIgc3Ryb2tlLXdpZHRoPSIxIi8+CiAgICA8Y2lyY2xlIGN4PSI1MCIgY3k9IjUwIiByPSIyNSIgZmlsbD0iI0YwRjBGMCIvPgogICAgPGxpbmUgeDE9IjUwIiB5MT0iNTAiIHgyPSI1MCIgeTI9IjMwIiBzdHJva2U9IiM1N2M0ZjgiIHN0cm9rZS13aWR0aD0iMyIgc3Ryb2tlLWxpbmVjYXA9InJvdW5kIi8+CiAgICA8bGluZSB4MT0iNTAiIHkxPSI1MCIgeDI9IjY1IiB5Mj0iNTAiIHN0cm9rZT0iIzU3YzRmOCIgc3Ryb2tlLXdpZHRoPSIzIiBzdHJva2UtbGluZWNhcD0icm91bmQiLz4KICAgIDx0ZXh0IHg9IjEwMCIgeT0iMzQiIGZvbnQtZmFtaWx5PSJBcmlhbCwgc2Fucy1zZXJpZiIgZm9udC1zaXplPSIxNCIgZmlsbD0iIzMzMzMzMyI+VGhlIG5leHQgY2VsbCBtYXkgdGFrZSBhIGZldyBtaW51dGVzIHRvIHJ1bi48L3RleHQ+Cjwvc3ZnPgo=\" alt=\"Time alert open medium\"/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6c7297a4-27c0-490b-bf1b-f2ba1a147dcb",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Wait until the endpoint has the status InService\n",
    "waiter = session.sagemaker_client.get_waiter('endpoint_in_service')\n",
    "waiter.wait(EndpointName=endpoint_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "95420ff1-d8cb-41b5-87bf-e011ca1c5f49",
   "metadata": {},
   "source": [
    "<img src=\"data:image/svg+xml;base64,Cjxzdmcgd2lkdGg9IjgwMCIgaGVpZ2h0PSI1MCIgdmlld0JveD0iMCAwIDgwMCA1MCIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj4KICAgIDxkZWZzPgogICAgICAgIDxsaW5lYXJHcmFkaWVudCBpZD0iZmFkZUdyYWRpZW50IiB4MT0iMCIgeDI9IjEiPgogICAgICAgICAgICA8c3RvcCBvZmZzZXQ9IjAlIiBzdG9wLWNvbG9yPSIjRjBGMEYwIi8+CiAgICAgICAgICAgIDxzdG9wIG9mZnNldD0iMTAwJSIgc3RvcC1jb2xvcj0iI0YwRjBGMCIgc3RvcC1vcGFjaXR5PSIwIi8+CiAgICAgICAgPC9saW5lYXJHcmFkaWVudD4KICAgICAgICA8bWFzayBpZD0iZmFkZU1hc2siPgogICAgICAgICAgICA8cmVjdCB4PSIwIiB5PSIwIiB3aWR0aD0iNzUwIiBoZWlnaHQ9IjUwIiBmaWxsPSJ3aGl0ZSIvPgogICAgICAgICAgICA8cmVjdCB4PSI3NTAiIHk9IjAiIHdpZHRoPSI1MCIgaGVpZ2h0PSI1MCIgZmlsbD0idXJsKCNmYWRlR3JhZGllbnQpIi8+CiAgICAgICAgPC9tYXNrPgogICAgPC9kZWZzPgogICAgPHBhdGggZD0iTTI1LDUwIFEwLDUwIDAsMjUgTDUwLDMgTDk3LDI1IEw3OTcsMjUgTDc5Nyw1MCBMMjUsNTAgWiIgZmlsbD0iI0YwRjBGMCIgc3Ryb2tlPSIjRTBFMEUwIiBzdHJva2Utd2lkdGg9IjEiIG1hc2s9InVybCgjZmFkZU1hc2spIi8+Cjwvc3ZnPgo=\" alt=\"Time alert close\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9faf03e3-8324-4843-9168-8f582557bf9b",
   "metadata": {},
   "source": [
    "You can see the deployed endpoint in the SageMaker console – click on the link constructed by the cell below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5d02df64-4d54-4f7a-9419-3cf577b321e4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Show the inference endpoint link\n",
    "display(\n",
    "    HTML('<b>See the SageMaker <a target=\"top\" href=\"https://{}.console.aws.amazon.com/sagemaker/home?region={}#/endpoints/{}\">inference endpoint</a></b>'.format(\n",
    "            region, region, endpoint_name))\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3bad12e8-264b-4558-aa50-8280a6d828e8",
   "metadata": {},
   "source": [
    "#### Predict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a6141539-44b4-4c42-830d-1862c2ee21a9",
   "metadata": {},
   "outputs": [],
   "source": [
    "predictions = predictor.predict(test_x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ce470852-0bb6-4868-a061-05022614e953",
   "metadata": {},
   "outputs": [],
   "source": [
    "predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d09c904e-3f27-472b-8e17-4d5daa4d0b4e",
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.crosstab(\n",
    "    index=test_y,\n",
    "    columns=np.round(predictions), \n",
    "    rownames=['actuals'], \n",
    "    colnames=['predictions']\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "16ffa17b-1077-47ce-b284-9514977a9607",
   "metadata": {},
   "outputs": [],
   "source": [
    "test_auc = roc_auc_score(test_y, predictions)\n",
    "print(f\"Test-auc: {test_auc:.4f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cfd3f1ac-aab7-4f23-a02e-0c9b2e3bcbfb",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "#### Save charts to the MLflow run\n",
    "You can use the [`mlflow.log_figure()`](https://mlflow.org/docs/latest/python_api/mlflow.html#mlflow.log_figure) logging method to save [matplotlib](https://matplotlib.org/3.3.4/api/_as_gen/matplotlib.figure.Figure.html) or [plotly](https://plotly.com/python-api-reference/generated/plotly.graph_objects.Figure.html) figures to a run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "098d3f32-ef0f-41da-a432-0b6e7cc1d359",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn import metrics\n",
    "from sklearn.metrics import RocCurveDisplay\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "def plot_confusion_matrix(\n",
    "    cm, class_names, title=\"Confusion matrix\", cmap=plt.cm.Blues, normalize=False\n",
    "):\n",
    "    if normalize:\n",
    "        cm = cm.astype(\"float\") / cm.sum(axis=1)[:, np.newaxis]\n",
    "\n",
    "    fig, ax = plt.subplots()\n",
    "    im = ax.imshow(cm, interpolation=\"nearest\", cmap=cmap)\n",
    "    ax.figure.colorbar(im, ax=ax)\n",
    "    ax.set(\n",
    "        xticks=np.arange(cm.shape[1]),\n",
    "        yticks=np.arange(cm.shape[0]),\n",
    "        ylim=(cm.shape[0] - 0.5, -0.5),\n",
    "        xticklabels=class_names,\n",
    "        yticklabels=class_names,\n",
    "        title=title,\n",
    "        ylabel=\"Ground truth label\",\n",
    "        xlabel=\"Predicted label\",\n",
    "    )\n",
    "\n",
    "    # Rotate the tick labels and set their alignment.\n",
    "    plt.setp(ax.get_xticklabels(), rotation=30, ha=\"right\", rotation_mode=\"anchor\")\n",
    "\n",
    "    # Loop over data dimensions and create text annotations.\n",
    "    fmt = \".2f\"\n",
    "    thresh = cm.max() / 2.0\n",
    "    for i in range(cm.shape[0]):\n",
    "        for j in range(cm.shape[1]):\n",
    "            ax.text(\n",
    "                j,\n",
    "                i,\n",
    "                format(cm[i, j], fmt),\n",
    "                ha=\"center\",\n",
    "                va=\"center\",\n",
    "                color=\"white\" if cm[i, j] > thresh else \"black\",\n",
    "            )\n",
    "    fig.tight_layout()\n",
    "    return ax, fig"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "914f414a-6494-4642-a35c-c4c80f05e764",
   "metadata": {},
   "outputs": [],
   "source": [
    "class_names = [\"no\", \"yes\"]\n",
    "confusion_matrix = metrics.confusion_matrix(test_y, np.round(predictions))\n",
    "ax, fig = plot_confusion_matrix(confusion_matrix, class_names)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4f19151b-a487-41ae-a7a8-6352762383e4",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f\"Log confusion matrix to the model {registered_model_version.name} version {registered_model_version.version}\")\n",
    "mlflow.set_experiment(experiment_name)\n",
    "with mlflow.start_run(run_id=registered_model_version.run_id):\n",
    "    mlflow.log_figure(fig, \"confusion_matrix.png\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9a19f3d4-70f7-4e39-869e-bb0e11e205eb",
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the presigned url to open the MLflow UI\n",
    "presigned_url = sm.create_presigned_mlflow_tracking_server_url(\n",
    "    TrackingServerName=mlflow_name,\n",
    "    ExpiresInSeconds=60,\n",
    "    SessionExpirationDurationInSeconds=1800\n",
    ")['AuthorizedUrl']\n",
    "\n",
    "mlflow_run_link = f\"{presigned_url.split('/auth')[0]}/#/experiments/1/runs/{registered_model_version.run_id}/artifacts\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "202b1bea-2a65-46bb-a601-9fa815e08175",
   "metadata": {},
   "outputs": [],
   "source": [
    "# first open the MLflow UI - you can close a new opened window\n",
    "display(Javascript('window.open(\"{}\");'.format(presigned_url)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fe867b2f-734c-4fb7-94f7-e9925950f06e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# second open the run page in the MLflow UI\n",
    "display(Javascript('window.open(\"{}\");'.format(mlflow_run_link)))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a6b32433-7af8-4796-9848-58b90f7cac00",
   "metadata": {},
   "source": [
    "#### Optional: deploy an endpoint from the estimator\n",
    "<div class=\"alert alert-info\">This section is optional – you don't need it for the further course of the workshop. To run this session you need to run a SageMaker built-in algorithm training job in the previous section <b>Optional: train with SageMaker built-in algorithms</b>.</div>\n",
    "\n",
    "If you have a trained estimator object, you can deploy the model in one line of code by using [`deploy()`](https://sagemaker.readthedocs.io/en/stable/api/training/estimators.html#sagemaker.estimator.Estimator.deploy) method of the estimator. This section shows how to do this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "feb0d417-99e8-480d-baa4-7ab18d9b5427",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# create a real-time endpoint from the trained estimator\n",
    "endpoint_name_from_estimator = f\"from-idea-to-prod-endpoint-estimator-{strftime('%d-%H-%M-%S', gmtime())}\"\n",
    "\n",
    "predictor_from_estimator = estimator.deploy(\n",
    "    initial_instance_count=1,\n",
    "    instance_type=\"ml.m5.large\",\n",
    "    wait=True,  # Remember, predictor.predict() won't work until deployment finishes!\n",
    "    # Turn on data capture, in case you want to experiment with monitoring:\n",
    "    data_capture_config=sagemaker.model_monitor.DataCaptureConfig(\n",
    "        enable_capture=True,\n",
    "        sampling_percentage=100,\n",
    "        destination_s3_uri=f\"s3://{bucket_name}/{bucket_prefix}/data-capture\",\n",
    "    ),\n",
    "    endpoint_name=endpoint_name_from_estimator,\n",
    "    serializer=sagemaker.serializers.CSVSerializer(),\n",
    "    deserializer=sagemaker.deserializers.CSVDeserializer(),\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cbfeb79b-014e-4784-a7bd-52f88b9957db",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# load test data as CSV files\n",
    "test_x = pd.read_csv(\"tmp/test_x.csv\", header=None)\n",
    "test_y = pd.read_csv(\"tmp/test_y.csv\", names=['y'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b5d420ff-2e3e-466a-b8e4-91a2f407f26d",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Predict using the real-time endpoint predictor\n",
    "predictions = np.array(predictor_from_estimator.predict(test_x.values), dtype=float).squeeze()\n",
    "predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ce21400a-6cbd-4817-869d-89e289fa165b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "test_results = pd.concat(\n",
    "    [\n",
    "        pd.Series(predictions, name=\"y_pred\", index=test_x.index),\n",
    "        test_x,\n",
    "    ],\n",
    "    axis=1,\n",
    ")\n",
    "test_results.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "91b8ef37-c50d-465d-bf54-2b104ace6fe7",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "pd.crosstab(\n",
    "    index=test_y['y'].values,\n",
    "    columns=np.round(predictions), \n",
    "    rownames=['actuals'], \n",
    "    colnames=['predictions']\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f994e5e0-b83f-4c39-b9fe-9e0c74b2b4f6",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "test_auc = roc_auc_score(test_y, test_results[\"y_pred\"])\n",
    "print(f\"Test-auc: {test_auc:.4f}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e3b0b514-0a8c-4fb3-96f8-e211ba282073",
   "metadata": {},
   "outputs": [],
   "source": [
    "# delete the endpoint to avoid cost\n",
    "predictor_from_estimator.delete_endpoint(delete_endpoint_config=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f2b56d63-c998-4ef1-ada2-e85ed1987e1e",
   "metadata": {},
   "source": [
    "## Optional: batch transform\n",
    "If you want to run off-line predictions on a large dataset or don't need a real-time endpoint, you can use SageMaker [batch-transform](https://docs.aws.amazon.com/sagemaker/latest/dg/batch-transform.html). This section demonstrates how to use [SageMaker batch transform](https://docs.aws.amazon.com/sagemaker/latest/dg/batch-transform.html).\n",
    "\n",
    "<div class=\"alert alert-info\">Run this session only if you trained an estimator with a SageMaker built-in XGBoost in the section <b>Optional: train with SageMaker built-in algorithms</b>.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bff1abd7-4ef7-4f27-a23a-1a254e823fa1",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "transform_s3_url = f\"s3://{bucket_name}/{bucket_prefix}/transform\"\n",
    "model_name = f\"from-idea-to-prod-transform-{strftime('%d-%H-%M-%S', gmtime())}\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7efc15f6-aa75-429f-9aed-c42572a06e7f",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\"> 💡 To create a transformer, use either <b>option 1</b> or <b>option 2</b>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cc425f73-03f6-481d-a337-d4f457a26ff6",
   "metadata": {},
   "source": [
    "### Option 1: create a batch transformer from the trained estimator\n",
    "You can use [`EstimatorBase.transformer()`](https://sagemaker.readthedocs.io/en/stable/api/training/estimators.html#sagemaker.estimator.EstimatorBase.transformer) to create a transformer for an estimator if you have a trained estimator. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bf85c452-ceca-402d-8cab-f1dafa3e4874",
   "metadata": {},
   "outputs": [],
   "source": [
    "try:\n",
    "    trained_estimator = estimator\n",
    "except NameError:\n",
    "    print(\"You don't have any trained estimator. Run SageMaker built-in algorithm training or use a training job name.\")\n",
    "\n",
    "trained_estimator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4a6d7f94-b194-4b9a-9430-a1d5b1ec055b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "transformer = trained_estimator.transformer(\n",
    "    instance_count=1,\n",
    "    instance_type=train_instance_type,\n",
    "    accept=\"text/csv\",\n",
    "    role=sm_role,\n",
    "    output_path=transform_s3_url,\n",
    "    model_name=model_name,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d00e05ab-c043-40fa-90dd-24d6850b7130",
   "metadata": {},
   "source": [
    "<div style=\"border: 4px solid coral; text-align: center; margin: auto;\">\n",
    "    <p style=\" text-align: center; margin: auto;\">Skip the Option 2 and go to the section <b>Run transform job</b>.\n",
    "    </p>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e49bf855-4b34-4ddb-ada6-736b2122ee18",
   "metadata": {},
   "source": [
    "### Option 2: load a model from a training job\n",
    "Alternatively, you can load a model from a model artifact produced by a training job. You create a transformer with that model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8ecb8929-90a0-4d6c-bba6-857ad8c2ac90",
   "metadata": {},
   "outputs": [],
   "source": [
    "if training_job_name is None:\n",
    "    print(\"You don't have saved trained job name. Run SageMaker built-in algorithm training or use a trained estimator.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e971d76c-8e06-4b94-9b25-6eaec8703622",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "model = session.create_model_from_job(\n",
    "    training_job_name=training_job_name, \n",
    "    name=model_name,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "50993f97-dce2-4084-a1de-22658fc8b669",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "transformer = sagemaker.transformer.Transformer(\n",
    "    model_name=model,\n",
    "    instance_count=1,\n",
    "    instance_type=train_instance_type,\n",
    "    accept=\"text/csv\",\n",
    "    assemble_with=\"Line\",\n",
    "    output_path=transform_s3_url,\n",
    "    base_transform_job_name=\"from-idea-to-prod-trasform\",\n",
    "    sagemaker_session=session,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "43fb1fac-ccfb-4c73-a215-013f991be48c",
   "metadata": {},
   "outputs": [],
   "source": [
    "!aws s3 cp ./tmp/test_x.csv {test_s3_url}/test_x.csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a7cac1f3-6465-4568-933a-f169bd6f68d5",
   "metadata": {},
   "outputs": [],
   "source": [
    "test_s3_url"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "60af22db-848d-44e8-987e-c22aedbd5354",
   "metadata": {},
   "source": [
    "### Run transform job\n",
    "\n",
    "<img src=\"data:image/svg+xml;base64,Cjxzdmcgd2lkdGg9IjgwMCIgaGVpZ2h0PSIxMjUiIHZpZXdCb3g9IjAgMCA4MDAgMTI1IiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPgogICAgPGRlZnM+CiAgICAgICAgPGxpbmVhckdyYWRpZW50IGlkPSJmYWRlR3JhZGllbnQiIHgxPSIwIiB4Mj0iMSI+CiAgICAgICAgICAgIDxzdG9wIG9mZnNldD0iMCUiIHN0b3AtY29sb3I9IiNGMEYwRjAiLz4KICAgICAgICAgICAgPHN0b3Agb2Zmc2V0PSIxMDAlIiBzdG9wLWNvbG9yPSIjRjBGMEYwIiBzdG9wLW9wYWNpdHk9IjAiLz4KICAgICAgICA8L2xpbmVhckdyYWRpZW50PgogICAgICAgIDxtYXNrIGlkPSJmYWRlTWFzayI+CiAgICAgICAgICAgIDxyZWN0IHg9IjAiIHk9IjAiIHdpZHRoPSI3NTAiIGhlaWdodD0iMTI1IiBmaWxsPSJ3aGl0ZSIvPgogICAgICAgICAgICA8cmVjdCB4PSI3NTAiIHk9IjAiIHdpZHRoPSI1MCIgaGVpZ2h0PSIxMjUiIGZpbGw9InVybCgjZmFkZUdyYWRpZW50KSIvPgogICAgICAgIDwvbWFzaz4KICAgIDwvZGVmcz4KICAgIDxwYXRoIGQ9Ik0zLDUwIEE1MCw1MCAwIDAgMSA1MywzIEw3OTcsMyBMNzk3LDk3IEw5Nyw5NyBMNTAsMTE1IEwzLDk3IFoiIGZpbGw9IiNGMEYwRjAiIHN0cm9rZT0iI0UwRTBFMCIgc3Ryb2tlLXdpZHRoPSIxIiBtYXNrPSJ1cmwoI2ZhZGVNYXNrKSIvPgogICAgPGNpcmNsZSBjeD0iNTAiIGN5PSI1MCIgcj0iMzAiIGZpbGw9IiM1N2M0ZjgiIHN0cm9rZT0iIzU3YzRmOCIgc3Ryb2tlLXdpZHRoPSIxIi8+CiAgICA8Y2lyY2xlIGN4PSI1MCIgY3k9IjUwIiByPSIyNSIgZmlsbD0iI0YwRjBGMCIvPgogICAgPGxpbmUgeDE9IjUwIiB5MT0iNTAiIHgyPSI1MCIgeTI9IjMwIiBzdHJva2U9IiM1N2M0ZjgiIHN0cm9rZS13aWR0aD0iMyIgc3Ryb2tlLWxpbmVjYXA9InJvdW5kIi8+CiAgICA8bGluZSB4MT0iNTAiIHkxPSI1MCIgeDI9IjY1IiB5Mj0iNTAiIHN0cm9rZT0iIzU3YzRmOCIgc3Ryb2tlLXdpZHRoPSIzIiBzdHJva2UtbGluZWNhcD0icm91bmQiLz4KICAgIDx0ZXh0IHg9IjEwMCIgeT0iMzQiIGZvbnQtZmFtaWx5PSJBcmlhbCwgc2Fucy1zZXJpZiIgZm9udC1zaXplPSIxNCIgZmlsbD0iIzMzMzMzMyI+VGhlIG5leHQgY2VsbCBtYXkgdGFrZSBhIGZldyBtaW51dGVzIHRvIHJ1bi48L3RleHQ+Cjwvc3ZnPgo=\" alt=\"Time alert open medium\"/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4b50377c-0196-41a3-9c8c-70ee2266c160",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "transform_job_name = f\"from-idea-to-prod-transform-{strftime('%d-%H-%M-%S', gmtime())}\"\n",
    "\n",
    "transformer.transform(    \n",
    "    data=f\"{test_s3_url}/test_x.csv\",\n",
    "    content_type=\"text/csv\",\n",
    "    split_type=\"Line\", \n",
    "    job_name=transform_job_name,\n",
    "    wait=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "edcfde81",
   "metadata": {},
   "source": [
    "<img src=\"data:image/svg+xml;base64,Cjxzdmcgd2lkdGg9IjgwMCIgaGVpZ2h0PSI1MCIgdmlld0JveD0iMCAwIDgwMCA1MCIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj4KICAgIDxkZWZzPgogICAgICAgIDxsaW5lYXJHcmFkaWVudCBpZD0iZmFkZUdyYWRpZW50IiB4MT0iMCIgeDI9IjEiPgogICAgICAgICAgICA8c3RvcCBvZmZzZXQ9IjAlIiBzdG9wLWNvbG9yPSIjRjBGMEYwIi8+CiAgICAgICAgICAgIDxzdG9wIG9mZnNldD0iMTAwJSIgc3RvcC1jb2xvcj0iI0YwRjBGMCIgc3RvcC1vcGFjaXR5PSIwIi8+CiAgICAgICAgPC9saW5lYXJHcmFkaWVudD4KICAgICAgICA8bWFzayBpZD0iZmFkZU1hc2siPgogICAgICAgICAgICA8cmVjdCB4PSIwIiB5PSIwIiB3aWR0aD0iNzUwIiBoZWlnaHQ9IjUwIiBmaWxsPSJ3aGl0ZSIvPgogICAgICAgICAgICA8cmVjdCB4PSI3NTAiIHk9IjAiIHdpZHRoPSI1MCIgaGVpZ2h0PSI1MCIgZmlsbD0idXJsKCNmYWRlR3JhZGllbnQpIi8+CiAgICAgICAgPC9tYXNrPgogICAgPC9kZWZzPgogICAgPHBhdGggZD0iTTI1LDUwIFEwLDUwIDAsMjUgTDUwLDMgTDk3LDI1IEw3OTcsMjUgTDc5Nyw1MCBMMjUsNTAgWiIgZmlsbD0iI0YwRjBGMCIgc3Ryb2tlPSIjRTBFMEUwIiBzdHJva2Utd2lkdGg9IjEiIG1hc2s9InVybCgjZmFkZU1hc2spIi8+Cjwvc3ZnPgo=\" alt=\"Time alert close\"/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1f11a902-1306-430f-8925-2ddd17741609",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "while sm.describe_transform_job(\n",
    "        TransformJobName=transformer._current_job_name\n",
    "    )[\"TransformJobStatus\"] != \"Completed\":\n",
    "    time.sleep(10)\n",
    "    print(f\"Wait until {transformer._current_job_name} completed\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "29c80898-b328-4413-bee2-34faf29f5f22",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "transformer.output_path"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "672524d0-d056-4a1d-b80a-b4e4822c8290",
   "metadata": {},
   "source": [
    "### Evaluate predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cc287aa8-de90-4de3-b59c-d7fc0e9aad2f",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "!aws s3 ls {transformer.output_path}/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ec638017-8bc3-498a-8b0d-26b5e85c0cde",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "!aws s3 cp {transformer.output_path}/test_x.csv.out tmp/predictions.csv\n",
    "!aws s3 cp $test_s3_url/test_y.csv tmp/test_y.csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4d5e4e16-4739-437c-8237-2083aae150be",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "predictions = pd.read_csv(\"tmp/predictions.csv\", names=[\"y_prob\"])\n",
    "test_y = pd.read_csv(\"tmp/test_y.csv\", names=['y'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "54f1497b-2723-4e37-a1e0-75117dcc91af",
   "metadata": {},
   "source": [
    "#### Crosstab"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5f5e150c-1bf7-414c-af9d-54ed108117aa",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "pd.crosstab(\n",
    "    index=test_y['y'].values,\n",
    "    columns=np.array(np.round(predictions), dtype=float).squeeze(), \n",
    "    rownames=['actuals'], \n",
    "    colnames=['predictions']\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bd11aa9b-9bcc-4e70-a508-93f02781fa73",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "test_auc = roc_auc_score(test_y, predictions)\n",
    "print(f\"Test-auc: {test_auc:.4f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d358952-5b3a-4690-b118-1677bd80fd5c",
   "metadata": {},
   "source": [
    "#### ROC curve"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eabaa6a7-08a2-4e21-95e2-17fea4c6e692",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "fpr, tpr, thresholds = metrics.roc_curve(test_y, predictions)\n",
    "roc_auc = metrics.auc(fpr, tpr)\n",
    "roc_display = metrics.RocCurveDisplay(fpr=fpr, tpr=tpr, roc_auc=roc_auc,estimator_name='Holdout/Test Data - ROC curve')\n",
    "roc_display.plot()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34c19463-4fb0-43bd-bb33-9f3a174c060e",
   "metadata": {},
   "source": [
    "#### Confusion matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "449ffe9b-1b7d-44d9-a9ce-c8d68c97de7e",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "class_names = [\"no\", \"yes\"]\n",
    "confusion_matrix = metrics.confusion_matrix(test_y, np.round(predictions))\n",
    "ax, fig = plot_confusion_matrix(confusion_matrix, class_names)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b90e2e7f-69fa-409b-a36f-341842d50146",
   "metadata": {},
   "source": [
    "#### Precision-recall curve"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c36302d4-5855-43cd-9d47-e5f2d50bfaf5",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import precision_recall_curve\n",
    "from sklearn.metrics import PrecisionRecallDisplay\n",
    "\n",
    "prec, recall, _ = precision_recall_curve(test_y, predictions)\n",
    "average_precision= metrics.average_precision_score(test_y, predictions)\n",
    "pr_display = PrecisionRecallDisplay(precision=prec, recall=recall, average_precision=average_precision, estimator_name='Holdout/Test Data - AUPRC curve')\n",
    "pr_display.plot()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "de6ee7d6-e8d7-40aa-bae6-8db6edbcc10d",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "## Explore experiments and model registry with the MLflow UI\n",
    "You can see all logged metrics, parameters, and artifacts in the MLflow UI. To launch the MLflow UI, choose **MLflow** in the **Application** pane of the Studio UI, select your MLflow server and choose **Open MLflow**:\n",
    "\n",
    "![](img/mlflow-open.png)\n",
    "\n",
    "In the MLflow UI you can browse experiments, runs, and registered models:\n",
    "\n",
    "![](img/experiment-mlflow-2.png)\n",
    "\n",
    "![](img/models-mlflow-2.png)\n",
    "\n",
    "![](img/model-fig-mlflow.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6949ec51-5f38-4432-9a5d-ed4e5350644f",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9d8a7389-9af4-4a0b-8ea7-376db0c2ed02",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "## Optional: Hyperparameter optimization (HPO)\n",
    "<div class=\"alert alert-info\">This section is optional – you don't need it for the further course of the workshop. You can move to the <b>Clean-up</b> section.</div>\n",
    "\n",
    "[Amazon SageMaker automatic model tuning](https://docs.aws.amazon.com/sagemaker/latest/dg/automatic-model-tuning.html), also called hyperparameter optimization (HPO), finds the best performing model against a defined objective metric by running many training jobs on the dataset using the algorithm and ranges of hyperparameters that you specify. SageMaker HPO supports random search, bayesian optimization, and [hyperband](https://docs.aws.amazon.com/sagemaker/latest/dg/automatic-model-tuning-how-it-works.html) as tuning strategies."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "543f54c4-11b1-418d-aab9-0fa52ae88835",
   "metadata": {},
   "outputs": [],
   "source": [
    "suffix = strftime('%d-%H-%M-%S', gmtime())\n",
    "hpo_experiment_name = f\"from-idea-to-prod-hpo-{suffix}\"\n",
    "registered_hpo_model_name = f\"from-idea-to-prod-hpo-model-{suffix}\"\n",
    "\n",
    "mlflow.set_experiment(hpo_experiment_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5230ec51-bab2-40ca-a5ce-3a31590d47a5",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# import required HPO objects\n",
    "from sagemaker.tuner import (\n",
    "    CategoricalParameter,\n",
    "    ContinuousParameter,\n",
    "    HyperparameterTuner,\n",
    "    IntegerParameter,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "531a6e93-5f57-43d1-851f-e6574b0d53ca",
   "metadata": {},
   "outputs": [],
   "source": [
    "# set up hyperparameter ranges\n",
    "hp_ranges = {\n",
    "    \"min_child_weight\": ContinuousParameter(1, 10),\n",
    "    \"max_depth\": IntegerParameter(1, 10),\n",
    "    \"alpha\": ContinuousParameter(0, 5),\n",
    "    \"eta\": ContinuousParameter(0, 1),\n",
    "    \"colsample_bytree\": ContinuousParameter(0, 1),\n",
    "}\n",
    "\n",
    "# set up the objective metric\n",
    "objective = \"validation:auc\"\n",
    "\n",
    "# set up the estimator in script mode\n",
    "hpo_estimator = XGBoost(\n",
    "    entry_point='train.py',\n",
    "    source_dir='./training',\n",
    "    framework_version=\"1.7-1\",  \n",
    "    hyperparameters=hyperparams,\n",
    "    role=sm_role,\n",
    "    instance_count=train_instance_count,\n",
    "    instance_type=train_instance_type,\n",
    "    output_path=output_s3_url,\n",
    "    base_job_name=\"from-idea-to-prod-training\",\n",
    "    environment={\n",
    "        'MLFLOW_TRACKING_ARN': mlflow_arn,\n",
    "        'MLFLOW_EXPERIMENT_NAME': hpo_experiment_name,\n",
    "        'USER': user_profile_name,\n",
    "        'REGION': region,\n",
    "    }\n",
    ")\n",
    "\n",
    "# instantiate a HPO object\n",
    "tuner = HyperparameterTuner(\n",
    "    estimator=hpo_estimator,  # the SageMaker estimator object\n",
    "    hyperparameter_ranges=hp_ranges,  # the range of hyperparameters\n",
    "    max_jobs=10,  # total number of HPO jobs\n",
    "    max_parallel_jobs=3,  # how many HPO jobs can run in parallel\n",
    "    strategy=\"Bayesian\",  # the internal optimization strategy of HPO\n",
    "    objective_metric_name=objective,  # the objective metric to be used for HPO\n",
    "    objective_type=\"Maximize\",  # maximize or minimize the objective metric\n",
    "    base_tuning_job_name=\"from-idea-to-prod-hpo\",\n",
    "    early_stopping_type=\"Auto\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7aab138e-5629-4a9d-830a-efb425a375bb",
   "metadata": {},
   "source": [
    "### Run the HPO\n",
    "Now run the HPO job. It takes about 10 minutes to complete."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9752f3de",
   "metadata": {},
   "source": [
    "<img src=\"data:image/svg+xml;base64,Cjxzdmcgd2lkdGg9IjgwMCIgaGVpZ2h0PSIxMjUiIHZpZXdCb3g9IjAgMCA4MDAgMTI1IiB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciPgogICAgPGRlZnM+CiAgICAgICAgPGxpbmVhckdyYWRpZW50IGlkPSJmYWRlR3JhZGllbnQiIHgxPSIwIiB4Mj0iMSI+CiAgICAgICAgICAgIDxzdG9wIG9mZnNldD0iMCUiIHN0b3AtY29sb3I9IiNGMEYwRjAiLz4KICAgICAgICAgICAgPHN0b3Agb2Zmc2V0PSIxMDAlIiBzdG9wLWNvbG9yPSIjRjBGMEYwIiBzdG9wLW9wYWNpdHk9IjAiLz4KICAgICAgICA8L2xpbmVhckdyYWRpZW50PgogICAgICAgIDxtYXNrIGlkPSJmYWRlTWFzayI+CiAgICAgICAgICAgIDxyZWN0IHg9IjAiIHk9IjAiIHdpZHRoPSI3NTAiIGhlaWdodD0iMTI1IiBmaWxsPSJ3aGl0ZSIvPgogICAgICAgICAgICA8cmVjdCB4PSI3NTAiIHk9IjAiIHdpZHRoPSI1MCIgaGVpZ2h0PSIxMjUiIGZpbGw9InVybCgjZmFkZUdyYWRpZW50KSIvPgogICAgICAgIDwvbWFzaz4KICAgIDwvZGVmcz4KICAgIDxwYXRoIGQ9Ik0zLDUwIEE1MCw1MCAwIDAgMSA1MywzIEw3OTcsMyBMNzk3LDk3IEw5Nyw5NyBMNTAsMTE1IEwzLDk3IFoiIGZpbGw9IiNGMEYwRjAiIHN0cm9rZT0iI0UwRTBFMCIgc3Ryb2tlLXdpZHRoPSIxIiBtYXNrPSJ1cmwoI2ZhZGVNYXNrKSIvPgogICAgPGNpcmNsZSBjeD0iNTAiIGN5PSI1MCIgcj0iMzAiIGZpbGw9IiM1N2M0ZjgiIHN0cm9rZT0iIzU3YzRmOCIgc3Ryb2tlLXdpZHRoPSIxIi8+CiAgICA8Y2lyY2xlIGN4PSI1MCIgY3k9IjUwIiByPSIyNSIgZmlsbD0iI0YwRjBGMCIvPgogICAgPGxpbmUgeDE9IjUwIiB5MT0iNTAiIHgyPSI1MCIgeTI9IjMwIiBzdHJva2U9IiM1N2M0ZjgiIHN0cm9rZS13aWR0aD0iMyIgc3Ryb2tlLWxpbmVjYXA9InJvdW5kIi8+CiAgICA8bGluZSB4MT0iNTAiIHkxPSI1MCIgeDI9IjY1IiB5Mj0iNTAiIHN0cm9rZT0iIzU3YzRmOCIgc3Ryb2tlLXdpZHRoPSIzIiBzdHJva2UtbGluZWNhcD0icm91bmQiLz4KICAgIDx0ZXh0IHg9IjEwMCIgeT0iMzQiIGZvbnQtZmFtaWx5PSJBcmlhbCwgc2Fucy1zZXJpZiIgZm9udC1zaXplPSIxNCIgZmlsbD0iIzMzMzMzMyI+VGhlIG5leHQgY2VsbCBtYXkgdGFrZSBhIGZldyBtaW51dGVzIHRvIHJ1bi4gUGxlYXNlIGJlIHBhdGllbnQuPC90ZXh0PgogICAgPHRleHQgeD0iMTAwIiB5PSI1NiIgZm9udC1mYW1pbHk9IkFyaWFsLCBzYW5zLXNlcmlmIiBmb250LXNpemU9IjE0IiBmaWxsPSIjMzMzMzMzIj5Zb3UgY2FuIHNhZmVseSBpZ25vcmUgdGhlIHdhcm5pbmcgbWVzc2FnZXMuPC90ZXh0Pgo8L3N2Zz4K\" alt=\"Time alert open medium\"/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "612a815f-c9f9-4e8e-9afb-ffd3b98a0f4b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "tuner.fit(\n",
    "    {\"train\": s3_input_train, \"validation\": s3_input_validation},\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0697a30f",
   "metadata": {},
   "source": [
    "<img src=\"data:image/svg+xml;base64,Cjxzdmcgd2lkdGg9IjgwMCIgaGVpZ2h0PSI1MCIgdmlld0JveD0iMCAwIDgwMCA1MCIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj4KICAgIDxkZWZzPgogICAgICAgIDxsaW5lYXJHcmFkaWVudCBpZD0iZmFkZUdyYWRpZW50IiB4MT0iMCIgeDI9IjEiPgogICAgICAgICAgICA8c3RvcCBvZmZzZXQ9IjAlIiBzdG9wLWNvbG9yPSIjRjBGMEYwIi8+CiAgICAgICAgICAgIDxzdG9wIG9mZnNldD0iMTAwJSIgc3RvcC1jb2xvcj0iI0YwRjBGMCIgc3RvcC1vcGFjaXR5PSIwIi8+CiAgICAgICAgPC9saW5lYXJHcmFkaWVudD4KICAgICAgICA8bWFzayBpZD0iZmFkZU1hc2siPgogICAgICAgICAgICA8cmVjdCB4PSIwIiB5PSIwIiB3aWR0aD0iNzUwIiBoZWlnaHQ9IjUwIiBmaWxsPSJ3aGl0ZSIvPgogICAgICAgICAgICA8cmVjdCB4PSI3NTAiIHk9IjAiIHdpZHRoPSI1MCIgaGVpZ2h0PSI1MCIgZmlsbD0idXJsKCNmYWRlR3JhZGllbnQpIi8+CiAgICAgICAgPC9tYXNrPgogICAgPC9kZWZzPgogICAgPHBhdGggZD0iTTI1LDUwIFEwLDUwIDAsMjUgTDUwLDMgTDk3LDI1IEw3OTcsMjUgTDc5Nyw1MCBMMjUsNTAgWiIgZmlsbD0iI0YwRjBGMCIgc3Ryb2tlPSIjRTBFMEUwIiBzdHJva2Utd2lkdGg9IjEiIG1hc2s9InVybCgjZmFkZU1hc2spIi8+Cjwvc3ZnPgo=\" alt=\"Time alert close\"/>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9eb7ee04-eee8-4e23-9e63-ad29cbbcf2d5",
   "metadata": {},
   "outputs": [],
   "source": [
    "tuner.describe()['HyperParameterTuningJobStatus']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b4a7395-f35b-4bca-8b4e-73f51bfad4cd",
   "metadata": {},
   "source": [
    "To see the details on the HPO job in the SageMaker console click on the link constructed by the cell below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2fe934e6-a68a-47e6-9bba-64f1a35af835",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Show the HPO job\n",
    "display(\n",
    "    HTML('<b>See the SageMaker <a target=\"top\" href=\"https://{}.console.aws.amazon.com/sagemaker/home?region={}#/hyper-tuning-jobs/{}\">HPO job</a></b>'.format(\n",
    "            region, region, tuner.latest_tuning_job.job_name))\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "24453c3a-e309-403a-8b32-a7735c726f95",
   "metadata": {},
   "source": [
    "### See HPO results\n",
    "Now get the results and see the best training job."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c2b0e7b2-cc3e-4900-8fa5-5550c5332633",
   "metadata": {},
   "outputs": [],
   "source": [
    "best_training_job = tuner.describe()['BestTrainingJob']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6c290b58-bf22-4ace-809b-b60c81e8af38",
   "metadata": {},
   "outputs": [],
   "source": [
    "best_training_job"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "386f87b9-2834-4f32-aa2f-d196c73fd093",
   "metadata": {},
   "source": [
    "### Register the best HPO model in the model registry\n",
    "Register the best HPO model with the hyperparameters, metrics, and model artifacts in the MLflow model registry."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1cb264d9-b41d-4a70-b4a4-12d88e463dd0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# find the run with the best HPO model\n",
    "best_hpo_model_run_id = mlflow.search_runs(\n",
    "    experiment_ids=[mlflow.get_experiment_by_name(hpo_experiment_name).experiment_id], \n",
    "    filter_string=f\"tags.mlflow.source.name LIKE '%{best_training_job['TrainingJobArn'].split('/')[-1]}%'\",\n",
    ")['run_id'][0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2366d8af-2f98-40db-84ad-6b50f0e4456e",
   "metadata": {},
   "outputs": [],
   "source": [
    "best_hpo_model_run_id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "261651e4-077a-487f-a7f8-4d283ea0d249",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Register the model\n",
    "model_uri = f\"runs:/{best_hpo_model_run_id}/model\"\n",
    "registered_hpo_model_version = mlflow.register_model(model_uri, registered_hpo_model_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2abee438-8068-46e4-bd0f-60a2bb50f4f4",
   "metadata": {},
   "source": [
    "As the next step you need to deploy the best HPO model and test it. You can choose one of the following options below."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7a94832-b635-41a1-9403-6e4b12cfdfbb",
   "metadata": {},
   "source": [
    "### Run local inference with the best HPO model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1e44ed71-3a44-434d-8073-3e660485ad8c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# get the model from the MLflow model registry\n",
    "model_uri = registered_hpo_model_version.source\n",
    "model = mlflow.xgboost.load_model(model_uri)\n",
    "\n",
    "# load data\n",
    "test_x = loadtxt(\"tmp/test_x.csv\", delimiter=\",\")\n",
    "test_y = loadtxt(\"tmp/test_y.csv\", delimiter=\",\")\n",
    "\n",
    "# predict\n",
    "dtest = xgb.DMatrix(test_x)\n",
    "\n",
    "predictions = np.array(model.predict(dtest), dtype=float).squeeze()\n",
    "predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "16362190-042e-4515-af23-4a44c24e3d82",
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.crosstab(\n",
    "    index=test_y,\n",
    "    columns=np.round(predictions), \n",
    "    rownames=['actuals'], \n",
    "    colnames=['predictions']\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e7bba99e-e648-4b73-9a26-5d5052f16f9e",
   "metadata": {},
   "outputs": [],
   "source": [
    "test_auc = roc_auc_score(test_y, predictions)\n",
    "print(f\"Test-auc: {test_auc:.4f}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9edd93b6-5a44-4998-8403-6d142b82ff95",
   "metadata": {},
   "source": [
    "There is only a small improvements for the model metrics. It can indicate, that the XGBoost model is already at it's limit. You might want to explore other model types to improve the prediction accuracy for this use case."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "64fd8435-9c7e-4b4f-87b2-13ee830d19e3",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f5493780-c392-4432-a34b-75f27ce984a5",
   "metadata": {},
   "source": [
    "## Clean-up\n",
    "To avoid charges, remove the hosted endpoint you created."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2de13080-5527-4de4-aacb-3eb50af0f92f",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "predictor.delete_endpoint(delete_endpoint_config=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34f720e0-e8d5-4cb5-93a6-6a4f2dafd523",
   "metadata": {},
   "source": [
    "## Continue with the step 3\n",
    "open the step 3 [notebook](03-sagemaker-pipeline.ipynb)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62b623d7-4b16-4c6f-96a8-ad92a77b1601",
   "metadata": {},
   "source": [
    "## Further development ideas for your real-world projects\n",
    "- [Manage experiments](https://docs.aws.amazon.com/sagemaker/latest/dg/mlflow.html)\n",
    "- Use [Amazon SageMaker Data Wrangler](https://docs.aws.amazon.com/sagemaker/latest/dg/data-wrangler.html) for creating a no-code or low-code visual data processing and feature engineering flow.\n",
    "- Try no-code [SageMaker Canvas](https://docs.aws.amazon.com/sagemaker/latest/dg/canvas.html) on your data to perform analysis and use automated ML to build models and generate predictions"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ba1fc9d-7a52-4d2f-b51e-0f9422119389",
   "metadata": {},
   "source": [
    "## Additional resources\n",
    "- [Using Docker containers with SageMaker](https://docs.aws.amazon.com/sagemaker/latest/dg/docker-containers.html)\n",
    "- [How to create and use a custom SageMaker container: SageMaker hands-on workshop](https://sagemaker-workshop.com/custom/containers.html)\n",
    "- [Amazon SageMaker Immersion Day](https://catalog.us-east-1.prod.workshops.aws/workshops/63069e26-921c-4ce1-9cc7-dd882ff62575/en-US)\n",
    "- [Targeting Direct Marketing with Amazon SageMaker XGBoost](https://github.com/aws-samples/amazon-sagemaker-immersion-day/blob/master/processing_xgboost.ipynb)\n",
    "- [Train a Machine Learning Model](https://aws.amazon.com/getting-started/hands-on/machine-learning-tutorial-train-a-model/)\n",
    "- [Deploy a Machine Learning Model to a Real-Time Inference Endpoint](https://aws.amazon.com/getting-started/hands-on/machine-learning-tutorial-deploy-model-to-real-time-inference-endpoint/)\n",
    "- [Amazon SageMaker 101 Workshop](https://catalog.us-east-1.prod.workshops.aws/workshops/0c6b8a23-b837-4e0f-b2e2-4a3ffd7d645b/en-US)\n",
    "- [Amazon SageMaker 101 Workshop code repository](https://github.com/aws-samples/sagemaker-101-workshop)\n",
    "- [Amazon SageMaker with XGBoost and Hyperparameter Tuning for Direct Marketing predictions](https://github.com/aws-samples/sagemaker-101-workshop/blob/main/builtin_algorithm_hpo_tabular/SageMaker%20XGBoost%20HPO.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2d7b222f-ee83-4c87-9019-cebcc7e56627",
   "metadata": {},
   "source": [
    "# Shutdown kernel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ae0b1f02-44b8-4749-a5e2-ce415fff7f78",
   "metadata": {},
   "outputs": [],
   "source": [
    "%%html\n",
    "\n",
    "<p><b>Shutting down your kernel for this notebook to release resources.</b></p>\n",
    "<button class=\"sm-command-button\" data-commandlinker-command=\"kernelmenu:shutdown\" style=\"display:none;\">Shutdown Kernel</button>\n",
    "        \n",
    "<script>\n",
    "try {\n",
    "    els = document.getElementsByClassName(\"sm-command-button\");\n",
    "    els[0].click();\n",
    "}\n",
    "catch(err) {\n",
    "    // NoOp\n",
    "}    \n",
    "</script>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "725f9d18-a550-46f6-9de8-9aeeed8171fd",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "availableInstances": [
   {
    "_defaultOrder": 0,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.t3.medium",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 1,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.t3.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 2,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.t3.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 3,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.t3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 4,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 5,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 6,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 7,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 8,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 9,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 10,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 11,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 12,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5d.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 13,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5d.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 14,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5d.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 15,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5d.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 16,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5d.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 17,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5d.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 18,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5d.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 19,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 20,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": true,
    "memoryGiB": 0,
    "name": "ml.geospatial.interactive",
    "supportedImageNames": [
     "sagemaker-geospatial-v1-0"
    ],
    "vcpuNum": 0
   },
   {
    "_defaultOrder": 21,
    "_isFastLaunch": true,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.c5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 22,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.c5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 23,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.c5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 24,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.c5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 25,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 72,
    "name": "ml.c5.9xlarge",
    "vcpuNum": 36
   },
   {
    "_defaultOrder": 26,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 96,
    "name": "ml.c5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 27,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 144,
    "name": "ml.c5.18xlarge",
    "vcpuNum": 72
   },
   {
    "_defaultOrder": 28,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.c5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 29,
    "_isFastLaunch": true,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g4dn.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 30,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g4dn.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 31,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g4dn.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 32,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g4dn.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 33,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g4dn.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 34,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g4dn.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 35,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 61,
    "name": "ml.p3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 36,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 244,
    "name": "ml.p3.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 37,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 488,
    "name": "ml.p3.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 38,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.p3dn.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 39,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.r5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 40,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.r5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 41,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.r5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 42,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.r5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 43,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.r5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 44,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.r5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 45,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.r5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 46,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.r5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 47,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 48,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 49,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 50,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 51,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 52,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 53,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.g5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 54,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.g5.48xlarge",
    "vcpuNum": 192
   },
   {
    "_defaultOrder": 55,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 56,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4de.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 57,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.trn1.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 58,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.trn1.32xlarge",
    "vcpuNum": 128
   },
   {
    "_defaultOrder": 59,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.trn1n.32xlarge",
    "vcpuNum": 128
   }
  ],
  "instance_type": "ml.t3.medium",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}