5 MLOps: Model Deployments.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "tags": []
   },
   "source": [
    "# MLOps \n",
    "\n",
    "This notebook is intended to help you get started managing your machine learning models and deployments in the H2O AI Cloud using python. \n",
    "\n",
    "**_This notebook assumes you have run 1 End to End Demo and have a project with a deployed model that exists with a specific name_**\n",
    "\n",
    "* **Product Documentation:** https://docs.h2o.ai/mlops/\n",
    "* **Python Documentation:** https://docs.h2o.ai/mlops/py-client-installing/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prerequistes\n",
    "\n",
    "Update the `h2o_ai_cloud.py` file with the connection parameters for your H2O AI Cloud environemnt:\n",
    "1. Login to your H2O AI Cloud environment\n",
    "1. Click your username or avatar in the H2O AI Cloud navigation bar\n",
    "1. Navigate to `CLI & API Access`\n",
    "1. Use the variables from the `Accessing H2O AI Cloud APIs` section to populate the parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "import json\n",
    "\n",
    "from h2o_ai_cloud import mlops_client\n",
    "import h2o_mlops_client\n",
    "\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Securely connect to the platform\n",
    "We first connect to the H2O AI Cloud using our platform token to create a token provider object. We can then use this object to log into Steam and other APIs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Connect to MLOps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Visit https://cloud.h2o.ai/auth/get-platform-token to get your platform token\n",
      "Enter your platform token: ········\n"
     ]
    }
   ],
   "source": [
    "mlops = mlops_client()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Projects"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create a project"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'project': {'created_time': datetime.datetime(2022, 7, 6, 1, 1, 47, 821112, tzinfo=tzutc()),\n",
       "             'description': 'Demo project for a tutorial',\n",
       "             'display_name': 'my_test_project',\n",
       "             'id': 'aafa75a7-1e4b-46d7-a0d9-b9595ed806c4',\n",
       "             'last_modified_time': datetime.datetime(2022, 7, 6, 1, 1, 47, 821112, tzinfo=tzutc()),\n",
       "             'owner_id': '6366bc29-5f35-4659-b1ee-165b875313f6'}}"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mlops.storage.project.create_project({\n",
    "    'project': {\n",
    "         'description': \"Demo project for a tutorial\",\n",
    "         'display_name': \"my_test_project\",\n",
    "    }\n",
    "})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### List all projects you have access to"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "aafa75a7-1e4b-46d7-a0d9-b9595ed806c4 my_test_project\n",
      "bbd95e60-74c2-4a95-856e-6d632dece23f Customer Churn\n",
      "d2169b15-50cf-44da-9761-0dbb2a5617cd Telco Churn Predictions\n"
     ]
    }
   ],
   "source": [
    "my_projects = mlops.storage.project.list_projects(body={}).project\n",
    "\n",
    "for p in my_projects:\n",
    "    print(p.id, p.display_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Delete a project "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "for p in my_projects:\n",
    "    if p.display_name == \"my_test_project\":\n",
    "        mlops.storage.project.delete_project({'project_id': p.id})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "bbd95e60-74c2-4a95-856e-6d632dece23f Customer Churn\n",
      "d2169b15-50cf-44da-9761-0dbb2a5617cd Telco Churn Predictions\n"
     ]
    }
   ],
   "source": [
    "for p in  mlops.storage.project.list_projects(body={}).project:\n",
    "    print(p.id, p.display_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Select a specific project to work with\n",
    "We have previously created this project using Driverless AI and added models to it in our first tutorial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "for p in my_projects:\n",
    "    if p.display_name == \"Telco Churn Predictions\":\n",
    "        project_id = p.id"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Experiments"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### List experiments\n",
    "Get a list of all experiments that are in our project"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "bc7539d4-fcc5-11ec-bbbf-8af6970414ab Default Baseline\n"
     ]
    }
   ],
   "source": [
    "my_project_experiments = mlops.storage.experiment.list_experiments({\n",
    "    'project_id': project_id,\n",
    "}).experiment\n",
    "\n",
    "for e in my_project_experiments:\n",
    "    print(e.id, e.display_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Select an experiment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'created_time': datetime.datetime(2022, 7, 6, 0, 53, 21, 186596, tzinfo=tzutc()),\n",
       " 'display_name': 'Default Baseline',\n",
       " 'id': 'bc7539d4-fcc5-11ec-bbbf-8af6970414ab',\n",
       " 'last_modified_time': datetime.datetime(2022, 7, 6, 0, 53, 21, 186596, tzinfo=tzutc()),\n",
       " 'metadata': None,\n",
       " 'owner_id': '6366bc29-5f35-4659-b1ee-165b875313f6',\n",
       " 'parameters': {'fold_column': '',\n",
       "                'target_column': 'Churn?',\n",
       "                'test_dataset_id': '',\n",
       "                'training_dataset_id': '',\n",
       "                'validation_dataset_id': '',\n",
       "                'weight_column': ''},\n",
       " 'statistics': {'training_duration': '117s'},\n",
       " 'status': 'EXPERIMENT_STATUS_UNSPECIFIED',\n",
       " 'tag': []}"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "experiment = mlops.storage.experiment.get_experiment({\n",
    "    'id': my_project_experiments[0].id, \n",
    "}).experiment\n",
    "experiment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Deployments"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Deployment environment \n",
    "`Dev` and `Prod` are deployment environment tags that you can use for your model deployements. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5ba7c218-8065-4e11-a819-c3bcbb991e1f DEV\n",
      "9ad0c266-5913-4616-9057-9daa3188252f PROD\n"
     ]
    }
   ],
   "source": [
    "my_project_deployment_environments = mlops.storage.deployment_environment.list_deployment_environments(body={\n",
    "    'project_id': project_id, \n",
    "}).deployment_environment\n",
    "\n",
    "for de in my_project_deployment_environments:\n",
    "    print(de.id, de.display_name)\n",
    "    if de.display_name == \"DEV\":\n",
    "        deployment_env_id = de.id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'created_time': datetime.datetime(2022, 7, 6, 0, 53, 19, 295718, tzinfo=tzutc()),\n",
       " 'deployment_target_name': 'kubernetes',\n",
       " 'display_name': 'DEV',\n",
       " 'id': '5ba7c218-8065-4e11-a819-c3bcbb991e1f',\n",
       " 'last_modified_time': datetime.datetime(2022, 7, 6, 0, 53, 19, 295718, tzinfo=tzutc()),\n",
       " 'project_id': 'd2169b15-50cf-44da-9761-0dbb2a5617cd'}"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "deployment_env = mlops.storage.deployment_environment.get_deployment_environment({\n",
    "    'deployment_environment_id': deployment_env_id\n",
    "}).deployment_environment\n",
    "deployment_env"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Deployment Types\n",
    "Deployment types help MLOps understand how traffic should be routed when new data is sent for predictions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['DEPLOYMENT_TYPE_UNSPECIFIED',\n",
       " 'SINGLE_MODEL',\n",
       " 'SHADOW_TRAFFIC',\n",
       " 'SPLIT_TRAFFIC']"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import h2o_mlops_client\n",
    "h2o_mlops_client.StorageDeploymentType().allowable_values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "deployed_model = mlops.storage.deployment_environment.deploy({\n",
    "    'deployment_environment_id': deployment_env_id,\n",
    "    'experiment_id': my_project_experiments[0].id,\n",
    "    'type': 'SINGLE_MODEL'\n",
    "})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Deployment health\n",
    "Wait until our deployment has gone from launching to healthy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "while mlops.deployer.deployment_status.get_deployment_status({\n",
    "    'deployment_id': deployed_model.deployment.id\n",
    "}).deployment_status.state != \"HEALTHY\":\n",
    "    pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "HEALTHY\n",
      "https://model.cloud.h2o.ai/f517404f-f1c9-48fb-96d7-24049b24cc7f/model/sample_request\n",
      "https://model.cloud.h2o.ai/f517404f-f1c9-48fb-96d7-24049b24cc7f/model/score\n"
     ]
    }
   ],
   "source": [
    "deployment_status = mlops.deployer.deployment_status.get_deployment_status({\n",
    "    'deployment_id': deployed_model.deployment.id\n",
    "}).deployment_status\n",
    "\n",
    "print(deployment_status.state)\n",
    "print(deployment_status.scorer.sample_request.url)\n",
    "print(deployment_status.scorer.score.url)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Make predictions\n",
    "Using the `https` library, we make predictions on new data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'fields': ['State',\n",
       "  'Account Length',\n",
       "  'Area Code',\n",
       "  \"Int'l Plan\",\n",
       "  'VMail Plan',\n",
       "  'VMail Message',\n",
       "  'Day Mins',\n",
       "  'Day Calls',\n",
       "  'Day Charge',\n",
       "  'Eve Mins',\n",
       "  'Eve Calls',\n",
       "  'Eve Charge',\n",
       "  'Night Mins',\n",
       "  'Night Calls',\n",
       "  'Night Charge',\n",
       "  'Intl Mins',\n",
       "  'Intl Calls',\n",
       "  'Intl Charge',\n",
       "  'CustServ Calls'],\n",
       " 'rows': [['text',\n",
       "   '0',\n",
       "   '0',\n",
       "   'text',\n",
       "   'text',\n",
       "   '0',\n",
       "   '0',\n",
       "   '0',\n",
       "   '0',\n",
       "   '0',\n",
       "   '0',\n",
       "   '0',\n",
       "   '0',\n",
       "   '0',\n",
       "   '0',\n",
       "   '0',\n",
       "   '0',\n",
       "   '0',\n",
       "   '0']]}"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample_request_as_text = requests.get(deployment_status.scorer.sample_request.url).text\n",
    "sample_request = json.loads(sample_request_as_text)\n",
    "sample_request"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['State',\n",
       " 'Account Length',\n",
       " 'Area Code',\n",
       " \"Int'l Plan\",\n",
       " 'VMail Plan',\n",
       " 'VMail Message',\n",
       " 'Day Mins',\n",
       " 'Day Calls',\n",
       " 'Day Charge',\n",
       " 'Eve Mins',\n",
       " 'Eve Calls',\n",
       " 'Eve Charge',\n",
       " 'Night Mins',\n",
       " 'Night Calls',\n",
       " 'Night Charge',\n",
       " 'Intl Mins',\n",
       " 'Intl Calls',\n",
       " 'Intl Charge',\n",
       " 'CustServ Calls']"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "fields = sample_request[\"fields\"]\n",
    "fields"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['text',\n",
       " '0',\n",
       " '0',\n",
       " 'text',\n",
       " 'text',\n",
       " '0',\n",
       " '0',\n",
       " '0',\n",
       " '0',\n",
       " '0',\n",
       " '0',\n",
       " '0',\n",
       " '0',\n",
       " '0',\n",
       " '0',\n",
       " '0',\n",
       " '0',\n",
       " '0',\n",
       " '0']"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sample_row = sample_request[\"rows\"][0]\n",
    "sample_row"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['10text',\n",
       " '100',\n",
       " '100',\n",
       " '10text',\n",
       " '10text',\n",
       " '100',\n",
       " '100',\n",
       " '100',\n",
       " '100',\n",
       " '100',\n",
       " '100',\n",
       " '100',\n",
       " '100',\n",
       " '100',\n",
       " '100',\n",
       " '100',\n",
       " '100',\n",
       " '100',\n",
       " '100']"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_row = ['10' + v for v in sample_row]\n",
    "new_row"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "new_predictions = requests.post(\n",
    "    url=deployment_status.scorer.score.url,\n",
    "    json={\n",
    "        'fields': fields,\n",
    "        'rows': [sample_row, new_row]\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Response [200]>"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'fields': ['Churn?.False.', 'Churn?.True.'],\n",
       " 'id': 'bc7539d4-fcc5-11ec-bbbf-8af6970414ab',\n",
       " 'score': [['0.96486217', '0.035137847'], ['0.55606866', '0.44393137']]}"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predictions_dict = json.loads(new_predictions.text)\n",
    "predictions_dict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Churn?.False.</th>\n",
       "      <th>Churn?.True.</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.96486217</td>\n",
       "      <td>0.035137847</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.55606866</td>\n",
       "      <td>0.44393137</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Churn?.False. Churn?.True.\n",
       "0    0.96486217  0.035137847\n",
       "1    0.55606866   0.44393137"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(predictions_dict[\"score\"], columns=predictions_dict[\"fields\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Delete a deployment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{}"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mlops.storage.deployment.undeploy({'deployment_id': deployed_model.deployment.id})"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": true,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}