03_multiclass_classification_cohere_classify.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Multiclass Classification w/ Cohere Classify"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Multiclass Classification` is a fundamental problem type in supervised learning where the goal is to classify instances into one or more classes. This notebook illustrates how to use `Cohere Classify` for multiclass classification."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1. Dataset Preparation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Note: you may need to restart the kernel to use updated packages.\n"
     ]
    }
   ],
   "source": [
    "%pip install --upgrade cohere datasets python-dotenv scikit-learn --quiet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import time\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import accuracy_score, classification_report\n",
    "\n",
    "from datasets import load_dataset\n",
    "from dotenv import find_dotenv, load_dotenv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The [AG_News](https://huggingface.co/datasets/ag_news) dataset is a collection of more than 1 million news articles. News articles have been gathered from more than 2000 news sources by ComeToMyHead in more than 1 year of activity. ComeToMyHead is an academic news search engine which has been running since July, 2004. The dataset is provided by the academic comunity for research purposes in data mining (clustering, classification), information retrieval (ranking, search), data compression, data streaming, and any other non-commercial activity. For more information, please refer to http://www.di.unipi.it/~gulli/AG_corpus_of_news_articles.html."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Dataset({\n",
       "    features: ['text', 'label'],\n",
       "    num_rows: 120000\n",
       "})"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ds = load_dataset(\"ag_news\", split=\"train\")\n",
    "ds"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's work with a smaller subset. We shuffle the dataset to maintain the balance among classes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Dataset({\n",
       "    features: ['text', 'label'],\n",
       "    num_rows: 1000\n",
       "})"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ds_sm = ds.shuffle(seed=42).select(range(1000))\n",
    "ds_sm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYEAAAEpCAYAAAB1Fp6nAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA0W0lEQVR4nO3deVhU9f4H8PewzbAjCAwIIoqKuN5cEFMEJcXMNL3u13BJU8FUrIyuikvFzbL8ZaZmBVZ6KzUtzQ1RXBI3yr1ccFdQQRBxYf38/vDhXEdQEVk979fznOfhfM/3nPM5Z4Z5z1lmRiMiAiIiUiWjyi6AiIgqD0OAiEjFGAJERCrGECAiUjGGABGRijEEiIhUjCFARKRiDAEiIhVjCBARqRhDgJ5ZZ8+ehUajQUxMTIWtMz4+HhqNBvHx8RW2zvIQExMDjUaD/fv3V8j6NBoNpk+fXiHrIkMMAZU6ePAgNBoNjh8/DgD49NNPUadOnWL77ty5E926dUOtWrWg0+lQu3Zt9OjRA8uWLavAitWrTp060Gg0xQ7BwcGVXd4jHThwAP/617/g7u4OrVYLe3t7BAUFITo6Gvn5+ZVdHgEwqewCqHLs2bMH9vb2aNCgAQAgISEBbdu2LdJv+fLl6N+/P1q0aIHx48ejRo0aOHPmDLZv347Fixdj0KBBFV16lebv7487d+7AzMysTJfbokULTJo0qUi7q6trma6nLH311VcYPXo0nJ2dMWTIENSvXx83b95EXFwcRowYgeTkZLz77ruVXabqMQRUau/evWjTpg00Gg2AeyEQHh5epN/06dPh4+OD3bt3F3lhu3r1aoXUWp0YGRlBp9OV+XJr1aqFf/3rX2W+3PKye/dujB49Gn5+fli3bh2sra2VaRMmTMD+/ftx5MiRSqyQCvF0kIqkp6cjNTUVqamp2LNnD5o0aYLU1FQcPXoUFy9eRP369ZGamoqsrCxlnqSkJLRu3brYd7ZOTk4G4x9//DHatWsHBwcHmJubo2XLllixYkWR+TQaDcLCwrB8+XL4+PjA3Nwcfn5+OHz4MABg0aJF8PLygk6nQ0BAAM6ePWswf0BAAJo0aYLExES0a9cO5ubm8PT0xMKFC0u0H/7++2/885//hL29PXQ6HVq1aoVff/3VoE9ubi5mzJiB+vXrQ6fTwcHBAe3bt0dsbOwjl13cNYHCeo8dO4bAwEBYWFigVq1amD17donqLalDhw5h6NChqFu3LnQ6HfR6PYYPH460tLQifS9duoQRI0bA1dUVWq0Wnp6eGDNmDHJycgz6ZWdnIzw8HI6OjrC0tMQrr7yCa9euPbaWGTNmQKPRYOnSpQYBUKhVq1YYOnToQ+c/d+4cxo4di4YNG8Lc3BwODg7o27dvkedCSR6nlJQUDBs2DG5ubtBqtXBxcUHPnj2LLEuteCSgIv/4xz9w7tw5ZfzIkSP4+OOPlfEePXoAAEJCQpSLqR4eHoiLi8PFixfh5ub2yOX/3//9H15++WUMHjwYOTk5+OGHH9C3b1+sXbsW3bt3N+i7Y8cO/PrrrwgNDQUAREVF4aWXXsLbb7+NL774AmPHjkV6ejpmz56N4cOHY8uWLQbzp6en48UXX0S/fv0wcOBA/PTTTxgzZgzMzMwwfPjwh9Z49OhRPP/886hVqxbeeecdWFpa4qeffkKvXr2wcuVKvPLKKwDuHQFFRUXhtddeQ5s2bZCZmYn9+/fjjz/+wAsvvPCYPV1Ueno6goOD0bt3b/Tr1w8rVqzA5MmT0bRpU3Tr1u2x8+fm5iI1NbVIu6WlJczNzQEAsbGxOH36NIYNGwa9Xo+jR4/iyy+/xNGjR7F7927lqO/y5cto06YNMjIyMGrUKHh7e+PSpUtYsWIFbt++bRD448aNQ40aNRAZGYmzZ89i7ty5CAsLw48//vjQWm/fvo24uDj4+/ujdu3aT7qrAAD79u3Drl27MGDAALi5ueHs2bNYsGABAgICcOzYMVhYWAAo2ePUp08fHD16FOPGjUOdOnVw9epVxMbG4vz58w+9DqYqQqqxc+dOiY2NlalTp4qJiYmsX79eYmNjpVu3btKqVSuJjY2V2NhYOXr0qDLP119/LQDEzMxMAgMDZerUqbJjxw7Jz88vsvzbt28bjOfk5EiTJk2kU6dOBu0ARKvVypkzZ5S2RYsWCQDR6/WSmZmptEdERAgAg74dO3YUADJnzhylLTs7W1q0aCFOTk6Sk5MjIiJnzpwRABIdHa3069y5szRt2lTu3r2rtBUUFEi7du2kfv36Slvz5s2le/fuj9mjRW3dulUAyNatW4vU++233xrUq9frpU+fPo9dpoeHhwAodoiKilL6Pbj/RUT++9//CgDZvn270vbqq6+KkZGR7Nu3r0j/goICERGJjo4WABIUFKS0iYhMnDhRjI2NJSMj46H1Hjx4UADI+PHjH7tthQBIZGTkI7clISGhyH583OOUnp4uAOSjjz4qcS1qw9NBKvL8888jKCgIWVlZaN26NYKDgxEUFITz58/jpZdeQlBQEIKCguDj46PMM3z4cGzYsAEBAQHYuXMnZs2ahQ4dOqB+/frYtWuXwfIL35EC99753rhxAx06dMAff/xRpJbOnTsbvAvz9fUFcO9d2/2nDwrbT58+bTC/iYkJXn/9dWXczMwMr7/+Oq5evYrExMRit//69evYsmUL+vXrh5s3byqnxtLS0tC1a1ecPHkSly5dAgDY2dnh6NGjOHny5CP3aUlZWVkZnNM3MzNDmzZtimzXw/j6+iI2NrbIMHDgQKXP/fv/7t27SE1NVS72Fz4GBQUFWL16NXr06IFWrVoVWU/h0UKhUaNGGbR16NAB+fn5BkeUD8rMzASAYk8DldT925Kbm4u0tDR4eXnBzs7O4Pn0uMfJ3NwcZmZmiI+PR3p6eqnreZYxBFTixo0byoteXFwcfH19kZqaihMnTuDo0aNo3rw5UlNTcePGjSLzdu3aFRs3bkRGRga2b9+O0NBQnDt3Di+99JLBxeG1a9eibdu20Ol0sLe3h6OjIxYsWFDsMh88TWBrawsAcHd3L7b9wX9gV1dXWFpaGrQV3un0sHO9p06dgohg6tSpcHR0NBgiIyMB/O9i98yZM5GRkYEGDRqgadOmeOutt3Do0KFil1sSbm5uRV5ga9SoUeIXppo1ayohff/g4eGh9Ll+/TrGjx8PZ2dnmJubw9HREZ6engCgPAbXrl1DZmYmmjRpUqL1Pvg41ahRA0DRx+N+NjY2AICbN2+WaB3FuXPnDqZNm6bcWlqzZk04OjoiIyPD4Pn0uMdJq9Xiww8/xPr16+Hs7Ax/f3/Mnj0bKSkppa7tWcMQUImePXsqL3iHDh3C3Llz4ejoiIYNGwIAXnnlFTg6OqJnz54PXYaFhQU6dOiAzz//HFOmTEF6ejrWr18P4N45/pdffhk6nQ5ffPEF1q1bh9jYWAwaNAhSzC+YGhsbF7uOh7UXt4wnVVBQAAB48803i31XHRsbCy8vLwD3bvVMSkrCN998gyZNmuCrr77Cc889h6+++qpU6y7P7SrUr18/LF68GKNHj8bPP/+MTZs2YcOGDQD+t+1PqjR1e3l5wcTERLnQXxrjxo3D+++/j379+uGnn37Cpk2bEBsbCwcHB4NtKcnjNGHCBJw4cQJRUVHQ6XSYOnUqGjVqhD///LPU9T1LeGFYJebMmYP09HQkJCRgxowZWLt2LUxMTDBv3jxcunQJ//nPfwD8753e4xSeSkhOTgYArFy5EjqdDhs3boRWq1X6RUdHl/GW3HP58mXcunXL4GjgxIkTAPDQi31169YFAJiamiIoKOix67C3t8ewYcMwbNgwZGVlwd/fH9OnT8drr7329BtQxtLT0xEXF4cZM2Zg2rRpSvuDp0kcHR1hY2NTrrdnWlhYoFOnTtiyZQsuXLhQ5OiuJFasWIGQkBDMmTNHabt79y4yMjKK9C3J41SvXj1MmjQJkyZNwsmTJ9GiRQvMmTMH33//fam28VnCIwGVaNmyJYKCgpCXl4cmTZoo1wOuXLlicHqhZcuWBvPFxcUVu7x169YBgHIkYWxsDI1GY/Ap0LNnz2L16tXlsj15eXlYtGiRMp6Tk4NFixbB0dGxyDYUcnJyQkBAABYtWqSE1/3uv/Xxwdsqrays4OXlhezs7DLagrJV+I79wXfoc+fONRg3MjJCr169sGbNmmK/EqKsjkwiIyMhIhgyZIjBLceFEhMTsWTJkofOb2xsXKSWefPmFfmU8eMep9u3b+Pu3bsGferVqwdra+sq+1hWNB4JqMzvv/+Odu3aAbj3zurPP/985Kc2e/bsCU9PT/To0QP16tXDrVu3sHnzZqxZswatW7dWbivt3r07PvnkEwQHB2PQoEG4evUq5s+fDy8vr6c6l/4wrq6u+PDDD3H27Fk0aNAAP/74Iw4cOIAvv/wSpqamD51v/vz5aN++PZo2bYqRI0eibt26uHLlChISEnDx4kUcPHgQAODj44OAgAC0bNkS9vb22L9/P1asWIGwsLAy35aSuHTpUrHvWq2srNCrVy/Y2Ngo57tzc3NRq1YtbNq0CWfOnCkyzwcffIBNmzahY8eOGDVqFBo1aoTk5GQsX74cO3fuhJ2d3VPX265dO8yfPx9jx46Ft7e3wSeG4+Pj8euvv+K999576PwvvfQSvvvuO9ja2sLHxwcJCQnYvHkzHBwcDPo97nE6ceIEOnfujH79+sHHxwcmJiZYtWoVrly5ggEDBjz1dj4TKu2+JKpweXl5YmVlJd99952I3LtlFIBcvXr1ofP897//lQEDBki9evXE3NxcdDqd+Pj4yL///W+DWzlF7t1OWr9+fdFqteLt7S3R0dESGRkpDz7NAEhoaKhBW+HtnA/eyld4y+Xy5cuVto4dO0rjxo1l//794ufnJzqdTjw8POTzzz8vdpn33yIqIpKUlCSvvvqq6PV6MTU1lVq1aslLL70kK1asUPq899570qZNG7GzsxNzc3Px9vaW999/X7n99GEedoto48aNi/QNCQkRDw+PRy5P5NG3iN4//8WLF+WVV14ROzs7sbW1lb59+8rly5eL3H4pInLu3Dl59dVXxdHRUbRardStW1dCQ0MlOztbRP53i+iDt5EWt32PkpiYKIMGDRJXV1cxNTWVGjVqSOfOnWXJkiUGtxk/WGN6eroMGzZMatasKVZWVtK1a1f5+++/xcPDQ0JCQpR+j3ucUlNTJTQ0VLy9vcXS0lJsbW3F19dXfvrppxLVrwYakTK8MkVUAQICApCamsqvHSAqA7wmQESkYgwBIiIVYwgQEakYrwkQEakYjwSIiFSMIUBEpGL8sBjufa/K5cuXYW1tXeRLvoiIqiMRwc2bN+Hq6gojo4e/32cI4N730JTm+02IiKq6CxcuPPIHoRgC+N/3nl+4cEH5GlwiouosMzMT7u7uj/1dB4YA/vdDGjY2NgwBInqmPO4UNy8MExGpGEOAiEjFGAJERCrGECAiUjGGABGRijEEiIhUjCFARKRiDAEiIhXjh8WIqpBFJ3+o7BIqxev1+aPvlYVHAkREKsYQICJSMYYAEZGKMQSIiFSMIUBEpGIMASIiFWMIEBGpGEOAiEjFGAJERCrGECAiUjGGABGRijEEiIhUrFJDICoqCq1bt4a1tTWcnJzQq1cvHD9+3KBPQEAANBqNwTB69GiDPufPn0f37t1hYWEBJycnvPXWW8jLy6vITSEiqpYq9VtEt23bhtDQULRu3Rp5eXl499130aVLFxw7dgyWlpZKv5EjR2LmzJnKuIWFhfJ3fn4+unfvDr1ej127diE5ORmvvvoqTE1N8cEHH1To9hARVTeVGgIbNmwwGI+JiYGTkxMSExPh7++vtFtYWECv1xe7jE2bNuHYsWPYvHkznJ2d0aJFC8yaNQuTJ0/G9OnTYWZmVq7bQERUnVWpawI3btwAANjb2xu0L126FDVr1kSTJk0QERGB27dvK9MSEhLQtGlTODs7K21du3ZFZmYmjh49Wux6srOzkZmZaTAQEalRlflRmYKCAkyYMAHPP/88mjRporQPGjQIHh4ecHV1xaFDhzB58mQcP34cP//8MwAgJSXFIAAAKOMpKSnFrisqKgozZswopy0hIqo+qkwIhIaG4siRI9i5c6dB+6hRo5S/mzZtChcXF3Tu3BlJSUmoV69eqdYVERGB8PBwZTwzMxPu7u6lK5yIqBqrEqeDwsLCsHbtWmzduhVubm6P7Ovr6wsAOHXqFABAr9fjypUrBn0Kxx92HUGr1cLGxsZgICJSo0oNARFBWFgYVq1ahS1btsDT0/Ox8xw4cAAA4OLiAgDw8/PD4cOHcfXqVaVPbGwsbGxs4OPjUy51ExE9Kyr1dFBoaCiWLVuGX375BdbW1so5fFtbW5ibmyMpKQnLli3Diy++CAcHBxw6dAgTJ06Ev78/mjVrBgDo0qULfHx8MGTIEMyePRspKSmYMmUKQkNDodVqK3PziIiqvEoNgQULFgC494Gw+0VHR2Po0KEwMzPD5s2bMXfuXNy6dQvu7u7o06cPpkyZovQ1NjbG2rVrMWbMGPj5+cHS0hIhISEGnyt4Wiv2XiuzZVUn/2zjWNklEFE5q9QQEJFHTnd3d8e2bdseuxwPDw+sW7eurMqiMpKxYV5ll1Ap7ILHVXYJRCVWZe4OIiIqrawVP1Z2CZXC6p/9n3oZVeLuICIiqhwMASIiFWMIEBGpGEOAiEjFGAJERCrGECAiUjGGABGRijEEiIhUjCFARKRiDAEiIhVjCBARqRhDgIhIxRgCREQqxhAgIlIxhgARkYoxBIiIVIwhQESkYgwBIiIVYwgQEakYQ4CISMUYAkREKsYQICJSMYYAEZGKMQSIiFSMIUBEpGIMASIiFWMIEBGpGEOAiEjFKjUEoqKi0Lp1a1hbW8PJyQm9evXC8ePHDfrcvXsXoaGhcHBwgJWVFfr06YMrV64Y9Dl//jy6d+8OCwsLODk54a233kJeXl5FbgoRUbVUqSGwbds2hIaGYvfu3YiNjUVubi66dOmCW7duKX0mTpyINWvWYPny5di2bRsuX76M3r17K9Pz8/PRvXt35OTkYNeuXViyZAliYmIwbdq0ytgkIqJqxaQyV75hwwaD8ZiYGDg5OSExMRH+/v64ceMGvv76ayxbtgydOnUCAERHR6NRo0bYvXs32rZti02bNuHYsWPYvHkznJ2d0aJFC8yaNQuTJ0/G9OnTYWZmVhmbRkRULVSpawI3btwAANjb2wMAEhMTkZubi6CgIKWPt7c3ateujYSEBABAQkICmjZtCmdnZ6VP165dkZmZiaNHj1Zg9URE1U+lHgncr6CgABMmTMDzzz+PJk2aAABSUlJgZmYGOzs7g77Ozs5ISUlR+twfAIXTC6cVJzs7G9nZ2cp4ZmZmWW0GEVG1UmWOBEJDQ3HkyBH88MMP5b6uqKgo2NraKoO7u3u5r5OIqCqqEiEQFhaGtWvXYuvWrXBzc1Pa9Xo9cnJykJGRYdD/ypUr0Ov1Sp8H7xYqHC/s86CIiAjcuHFDGS5cuFCGW0NEVH1UagiICMLCwrBq1Sps2bIFnp6eBtNbtmwJU1NTxMXFKW3Hjx/H+fPn4efnBwDw8/PD4cOHcfXqVaVPbGwsbGxs4OPjU+x6tVotbGxsDAYiIjWq1GsCoaGhWLZsGX755RdYW1sr5/BtbW1hbm4OW1tbjBgxAuHh4bC3t4eNjQ3GjRsHPz8/tG3bFgDQpUsX+Pj4YMiQIZg9ezZSUlIwZcoUhIaGQqvVVubmERFVeZUaAgsWLAAABAQEGLRHR0dj6NChAIBPP/0URkZG6NOnD7Kzs9G1a1d88cUXSl9jY2OsXbsWY8aMgZ+fHywtLRESEoKZM2dW1GYQEVVblRoCIvLYPjqdDvPnz8f8+fMf2sfDwwPr1q0ry9KIiFShSlwYJiKiysEQICJSMYYAEZGKMQSIiFSMIUBEpGIMASIiFWMIEBGpGEOAiEjFGAJERCrGECAiUjGGABGRijEEiIhUjCFARKRiDAEiIhVjCBARqRhDgIhIxRgCREQqxhAgIlIxhgARkYoxBIiIVIwhQESkYgwBIiIVYwgQEalYqUKgU6dOyMjIKNKemZmJTp06PW1NRERUQUoVAvHx8cjJySnSfvfuXezYseOpiyIiooph8iSdDx06pPx97NgxpKSkKOP5+fnYsGEDatWqVXbVERFRuXqiEGjRogU0Gg00Gk2xp33Mzc0xb968MiuOiIjK1xOFwJkzZyAiqFu3Lvbu3QtHR0dlmpmZGZycnGBsbFzmRRIRUfl4ohDw8PAAABQUFJRLMUREVLGeKATud/LkSWzduhVXr14tEgrTpk176sKIiKj8leruoMWLF6NRo0aYNm0aVqxYgVWrVinD6tWrS7yc7du3o0ePHnB1dYVGoyky79ChQ5VrEIVDcHCwQZ/r169j8ODBsLGxgZ2dHUaMGIGsrKzSbBYRkeqU6kjgvffew/vvv4/Jkyc/1cpv3bqF5s2bY/jw4ejdu3exfYKDgxEdHa2Ma7Vag+mDBw9GcnIyYmNjkZubi2HDhmHUqFFYtmzZU9VGRKQGpQqB9PR09O3b96lX3q1bN3Tr1u2RfbRaLfR6fbHT/vrrL2zYsAH79u1Dq1atAADz5s3Diy++iI8//hiurq5PXSMR0bOsVKeD+vbti02bNpV1LcWKj4+Hk5MTGjZsiDFjxiAtLU2ZlpCQADs7OyUAACAoKAhGRkbYs2dPhdRHRFSdlepIwMvLC1OnTsXu3bvRtGlTmJqaGkx/4403yqS44OBg9O7dG56enkhKSsK7776Lbt26ISEhAcbGxkhJSYGTk5PBPCYmJrC3tzf4INuDsrOzkZ2drYxnZmaWSb1ERNVNqULgyy+/hJWVFbZt24Zt27YZTNNoNGUWAgMGDFD+btq0KZo1a4Z69eohPj4enTt3LvVyo6KiMGPGjLIokYioWitVCJw5c6as6yiRunXrombNmjh16hQ6d+4MvV6Pq1evGvTJy8vD9evXH3odAQAiIiIQHh6ujGdmZsLd3b3c6iYiqqqq1VdJX7x4EWlpaXBxcQEA+Pn5ISMjA4mJiUqfLVu2oKCgAL6+vg9djlarhY2NjcFARKRGpToSGD58+COnf/PNNyVaTlZWFk6dOqWMnzlzBgcOHIC9vT3s7e0xY8YM9OnTB3q9HklJSXj77bfh5eWFrl27AgAaNWqE4OBgjBw5EgsXLkRubi7CwsIwYMAA3hlERFQCpb5F9H65ubk4cuQIMjIynuj3BPbv34/AwEBlvPAUTUhICBYsWIBDhw5hyZIlyMjIgKurK7p06YJZs2YZfFZg6dKlCAsLQ+fOnWFkZIQ+ffrgs88+K81mERGpTqlCYNWqVUXaCgoKMGbMGNSrV6/EywkICICIPHT6xo0bH7sMe3t7fjCMiKiUyuyagJGREcLDw/Hpp5+W1SKJiKiclemF4aSkJOTl5ZXlIomIqByV6nTQ/bdXAoCIIDk5Gb/99htCQkLKpDAiIip/pQqBP//802DcyMgIjo6OmDNnzmPvHCIioqqjVCGwdevWsq6DiIgqQal/VAYArl27huPHjwMAGjZsaPBzk0REVPWV6sLwrVu3MHz4cLi4uMDf3x/+/v5wdXXFiBEjcPv27bKukYiIykmpQiA8PBzbtm3DmjVrkJGRgYyMDPzyyy/Ytm0bJk2aVNY1EhFROSnV6aCVK1dixYoVCAgIUNpefPFFmJubo1+/fliwYEFZ1UdEROWoVEcCt2/fhrOzc5F2Jycnng4iIqpGShUCfn5+iIyMxN27d5W2O3fuYMaMGfDz8yuz4oiIqHyV6nTQ3LlzERwcDDc3NzRv3hwAcPDgQWi12gr72UkiInp6pQqBpk2b4uTJk1i6dCn+/vtvAMDAgQMxePBgmJubl2mBRERUfkoVAlFRUXB2dsbIkSMN2r/55htcu3YNkydPLpPiiIiofJXqmsCiRYvg7e1dpL1x48ZYuHDhUxdFREQVo1QhkJKSovzE4/0cHR2RnJz81EUREVHFKFUIuLu74/fffy/S/vvvv/NnHYmIqpFSXRMYOXIkJkyYgNzcXOXnJOPi4vD222/zE8NERNVIqULgrbfeQlpaGsaOHYucnBwAgE6nw+TJkxEREVGmBRIRUfkpVQhoNBp8+OGHmDp1Kv766y+Ym5ujfv36Bj8AT0REVd9TfZW0lZUVWrduXVa1EBFRBSvT3xgmIqLqhSFARKRiDAEiIhVjCBARqRhDgIhIxRgCREQqxhAgIlIxhgARkYoxBIiIVKxSQ2D79u3o0aMHXF1dodFosHr1aoPpIoJp06bBxcUF5ubmCAoKwsmTJw36XL9+HYMHD4aNjQ3s7OwwYsQIZGVlVeBWEBFVX5UaArdu3ULz5s0xf/78YqfPnj0bn332GRYuXIg9e/bA0tISXbt2NfiB+8GDB+Po0aOIjY3F2rVrsX37dowaNaqiNoGIqFp7qu8OelrdunVDt27dip0mIpg7dy6mTJmCnj17AgC+/fZbODs7Y/Xq1RgwYAD++usvbNiwAfv27UOrVq0AAPPmzcOLL76Ijz/+mL9tQET0GFX2msCZM2eQkpKCoKAgpc3W1ha+vr5ISEgAACQkJMDOzk4JAAAICgqCkZER9uzZ89BlZ2dnIzMz02AgIlKjKhsCKSkpAABnZ2eDdmdnZ2VaSkoKnJycDKabmJjA3t5e6VOcqKgo2NraKoO7u3sZV09EVD1U2RAoTxEREbhx44YyXLhwobJLIiKqFFU2BPR6PQDgypUrBu1XrlxRpun1ely9etVgel5eHq5fv670KY5Wq4WNjY3BQESkRlU2BDw9PaHX6xEXF6e0ZWZmYs+ePfDz8wMA+Pn5ISMjA4mJiUqfLVu2oKCgAL6+vhVeMxFRdVOpdwdlZWXh1KlTyviZM2dw4MAB2Nvbo3bt2pgwYQLee+891K9fH56enpg6dSpcXV3Rq1cvAECjRo0QHByMkSNHYuHChcjNzUVYWBgGDBjAO4OIiEqgUkNg//79CAwMVMbDw8MBACEhIYiJicHbb7+NW7duYdSoUcjIyED79u2xYcMG6HQ6ZZ6lS5ciLCwMnTt3hpGREfr06YPPPvuswreFiKg6qtQQCAgIgIg8dLpGo8HMmTMxc+bMh/axt7fHsmXLyqM8IqJnXpW9JkBEROWPIUBEpGIMASIiFWMIEBGpGEOAiEjFGAJERCrGECAiUjGGABGRijEEiIhUjCFARKRiDAEiIhVjCBARqRhDgIhIxRgCREQqxhAgIlIxhgARkYoxBIiIVIwhQESkYgwBIiIVYwgQEakYQ4CISMUYAkREKsYQICJSMYYAEZGKMQSIiFSMIUBEpGIMASIiFWMIEBGpGEOAiEjFqnQITJ8+HRqNxmDw9vZWpt+9exehoaFwcHCAlZUV+vTpgytXrlRixURE1UuVDgEAaNy4MZKTk5Vh586dyrSJEydizZo1WL58ObZt24bLly+jd+/elVgtEVH1YlLZBTyOiYkJ9Hp9kfYbN27g66+/xrJly9CpUycAQHR0NBo1aoTdu3ejbdu2FV0qEVG1U+WPBE6ePAlXV1fUrVsXgwcPxvnz5wEAiYmJyM3NRVBQkNLX29sbtWvXRkJCQmWVS0RUrVTpIwFfX1/ExMSgYcOGSE5OxowZM9ChQwccOXIEKSkpMDMzg52dncE8zs7OSElJeeRys7OzkZ2drYxnZmaWR/lERFVelQ6Bbt26KX83a9YMvr6+8PDwwE8//QRzc/NSLzcqKgozZswoixKJiKq1Kn866H52dnZo0KABTp06Bb1ej5ycHGRkZBj0uXLlSrHXEO4XERGBGzduKMOFCxfKsWoioqqrWoVAVlYWkpKS4OLigpYtW8LU1BRxcXHK9OPHj+P8+fPw8/N75HK0Wi1sbGwMBiIiNarSp4PefPNN9OjRAx4eHrh8+TIiIyNhbGyMgQMHwtbWFiNGjEB4eDjs7e1hY2ODcePGwc/Pj3cGERGVUJUOgYsXL2LgwIFIS0uDo6Mj2rdvj927d8PR0REA8Omnn8LIyAh9+vRBdnY2unbtii+++KKSqyYiqj6qdAj88MMPj5yu0+kwf/58zJ8/v4IqIiJ6tlSrawJERFS2GAJERCrGECAiUjGGABGRijEEiIhUjCFARKRiDAEiIhVjCBARqRhDgIhIxRgCREQqxhAgIlIxhgARkYoxBIiIVIwhQESkYgwBIiIVYwgQEakYQ4CISMUYAkREKsYQICJSMYYAEZGKMQSIiFSMIUBEpGIMASIiFWMIEBGpGEOAiEjFGAJERCrGECAiUjGGABGRijEEiIhUjCFARKRiz0wIzJ8/H3Xq1IFOp4Ovry/27t1b2SUREVV5z0QI/PjjjwgPD0dkZCT++OMPNG/eHF27dsXVq1cruzQioirtmQiBTz75BCNHjsSwYcPg4+ODhQsXwsLCAt98801ll0ZEVKWZVHYBTysnJweJiYmIiIhQ2oyMjBAUFISEhIRi58nOzkZ2drYyfuPGDQBAZmZmsf1vZ90sw4qrj8xM7dPNf+tOGVVSvRg95HlUEneybpdhJdXHw/73Sirrtjr3W8Ej9lvhPhWRRy6j2odAamoq8vPz4ezsbNDu7OyMv//+u9h5oqKiMGPGjCLt7u7u5VIjqc3kyi6g2pmIEZVdQjU1/LE9bt68CVtb24dOr/YhUBoREREIDw9XxgsKCnD9+nU4ODhAo9FUYmWGMjMz4e7ujgsXLsDGxqayy6k2uN+eHPdZ6VTl/SYiuHnzJlxdXR/Zr9qHQM2aNWFsbIwrV64YtF+5cgV6vb7YebRaLbRaw1MddnZ25VXiU7OxsalyT7DqgPvtyXGflU5V3W+POgIoVO0vDJuZmaFly5aIi4tT2goKChAXFwc/P79KrIyIqOqr9kcCABAeHo6QkBC0atUKbdq0wdy5c3Hr1i0MGzassksjIqrSnokQ6N+/P65du4Zp06YhJSUFLVq0wIYNG4pcLK5utFotIiMji5y6okfjfnty3Gel8yzsN4087v4hIiJ6ZlX7awJERFR6DAEiIhVjCBARqRhDoJqbPn06WrRo8cg+Q4cORa9evSqkHqJnWXx8PDQaDTIyMiq7lDLDECgHCxcuhLW1NfLy8pS2rKwsmJqaIiAgwKBv4ZMqKSmpgqusWq5du4YxY8agdu3a0Gq10Ov16Nq1K37//fdyX3edOnUwd+7ccl9PZYiJiam0D0JWxpuPoUOHQqPRKIODgwOCg4Nx6NChMll+u3btkJycXKIPYVUXDIFyEBgYiKysLOzfv19p27FjB/R6Pfbs2YO7d+8q7Vu3bkXt2rVRr169J1qHiBiETHXXp08f/Pnnn1iyZAlOnDiBX3/9FQEBAUhLSyu3debk5JTbssvS0wRk//79ceLEiSLtS5YsgZubm8ELZnFDTExMOWxR+QoODkZycjKSk5MRFxcHExMTvPTSS2WybDMzM+j1+ir19TJPiyFQDho2bAgXFxfEx8crbfHx8ejZsyc8PT2xe/dug/bAwEBkZ2fjjTfegJOTE3Q6Hdq3b499+/YZ9NNoNFi/fj1atmwJrVaLnTt3Fll3fn4+wsPDYWdnBwcHB7z99tuP/RbBypaRkYEdO3bgww8/RGBgIDw8PNCmTRtERETg5ZdfBgBoNBosWLAA3bp1g7m5OerWrYsVK1YYLOfw4cPo1KkTzM3N4eDggFGjRiErK0uZXvjO9P3334erqysaNmyIgIAAnDt3DhMnTlRe+ADg3Llz6NGjB2rUqAFLS0s0btwY69atq7idcp+nCUhzc3M4OTkVaf/ll18wbtw45cUyOTkZkyZNQuPGjQ3a+vfvXx6bVK4Kg1Kv16NFixZ45513cOHCBVy7dq3Y0zkHDhyARqPB2bNnATz6sX9w/sIjrY0bN6JRo0awsrJSQuh+X331FRo1agSdTgdvb2988cUXyrScnByEhYXBxcUFOp0OHh4eiIqKAnDvzd706dOVNwCurq544403ynaHCZWLQYMGSZcuXZTx1q1by/Lly2X06NEybdo0ERG5ffu2aLVaiYmJkTfeeENcXV1l3bp1cvToUQkJCZEaNWpIWlqaiIhs3bpVAEizZs1k06ZNcurUKUlLS5PIyEhp3ry5sp4PP/xQatSoIStXrpRjx47JiBEjxNraWnr27FmRm/9EcnNzxcrKSiZMmCB3794ttg8AcXBwkMWLF8vx48dlypQpYmxsLMeOHRMRkaysLHFxcZHevXvL4cOHJS4uTjw9PSUkJERZRkhIiFhZWcmQIUPkyJEjcuTIEUlLSxM3NzeZOXOmJCcnS3JysoiIdO/eXV544QU5dOiQJCUlyZo1a2Tbtm3lvi8elJ6eLgAkPj7+kX1GjRolTk5OotVqpXHjxrJmzRoREYmOjhZbW1uD/nfu3BFLS0v566+/DNoffC7l5+fLBx98IHXq1BGdTifNmjWT5cuXG8xz5MgR6d69u1hbW4uVlZW0b99eTp06JSL39nfPnj3lo48+Er1eL/b29jJ27FjJycl5ij3yaIXrLHTz5k15/fXXxcvLS/Lz85X/o/T0dKXPn3/+KQDkzJkzIvLox/7B+aOjo8XU1FSCgoJk3759kpiYKI0aNZJBgwYpy//+++/FxcVFVq5cKadPn5aVK1eKvb29xMTEiIjIRx99JO7u7rJ9+3Y5e/as7NixQ5YtWyYiIsuXLxcbGxtZt26dnDt3Tvbs2SNffvllme4zhkA5Wbx4sVhaWkpubq5kZmaKiYmJXL16VZYtWyb+/v4iIhIXFycA5OzZs2JqaipLly5V5s/JyRFXV1eZPXu2iPzvybd69WqD9Tz4j+vi4qLMI3LvBdbNza1Kh4CIyIoVK6RGjRqi0+mkXbt2EhERIQcPHlSmA5DRo0cbzOPr6ytjxowREZEvv/xSatSoIVlZWcr03377TYyMjCQlJUVE7r1AODs7S3Z2tsFyPDw85NNPPzVoa9q0qUyfPr0sN7FUHheQ+fn50rZtW2ncuLFs2rRJedFat26diBQfAmvXrpUGDRoUWdaDz6X33ntPvL29ZcOGDZKUlCTR0dGi1WqVQLp48aLY29tL7969Zd++fXL8+HH55ptv5O+//xaRe/vbxsZGRo8eLX/99ZesWbNGLCwsyvxF7H4hISFibGwslpaWYmlpKQDExcVFEhMTRaToi7hI0RB41GNfXAgAUIJPRGT+/Pni7OysjNerV095US80a9Ys8fPzExGRcePGSadOnaSgoKDI+ubMmSMNGjQo1+BkCJSTkydPCgDZtWuX/Pbbb+Lj4yMiIpcuXRKtVit37tyRqVOnSt26deXgwYNKGNyvV69eMmzYMBH535Pv4sWLBn3u/8fNyMgQAEXesfbq1avKh4DIvXeomzZtkpkzZ4qfn58YGxtLdHS0iNwLgSVLlhj0nzBhggQEBIiIyMSJE5W/Cz24P0JCQiQoKKjIeosLgcWLF4uJiYm0a9dOpk2bZhBIFe1RAblx40YxMjKS48ePFztvcSEwcuRIefPNN4v0vf+5dPfuXbGwsJBdu3YZ9BkxYoQMHDhQREQiIiLE09PzoS9QISEh4uHhIXl5eUpb3759pX///iXa7tIofIxPnjwpJ0+elL1798rQoUPFyclJzp49W6IQeNRjX1wIWFhYGNTw888/i0ajEZF7R6gAxNzcXAkmS0tL0Wq14uTkJCIiiYmJYm9vL/Xr15dx48bJxo0blWWdP39e3N3dxc3NTV577TX5+eefJTc3t0z3Ga8JlBMvLy+4ublh69at2Lp1Kzp27AgAcHV1hbu7O3bt2oWtW7eiU6dOT7RcS0vL8ii3StDpdHjhhRcwdepU7Nq1C0OHDkVkZGSZrqOk+++1117D6dOnMWTIEBw+fBitWrXCvHnzyrSWkurTpw8uX76MX3/9FcHBwYiPj8dzzz2HmJgYHDhwAG5ubmjQoEGJliUiWLNmjXKt5WFOnTqF27dv44UXXoCVlZUyfPvtt8qdbAcOHECHDh1gamr60OU0btwYxsbGyriLi0u5//a3paUlvLy84OXlhdatW+Orr77CrVu3sHjxYhgZ3XvJk/uuk+Xm5hrM/6SP/YPbr9FolOUXXpNavHgxDhw4oAxHjhxRrg0+99xzOHPmDGbNmoU7d+6gX79++Oc//wng3g9dHT9+HF988QXMzc0xduxY+Pv7F6n5aTAEylFgYCDi4+MRHx9vcGuov78/1q9fj7179yIwMBD16tWDmZmZwd0eubm52LdvH3x8fEq8PltbW7i4uGDPnj1KW15eHhITE8tkeyqaj48Pbt26pYzff0G9cLxRo0YAgEaNGuHgwYMG/X///XcYGRmhYcOGj1yPmZkZ8vPzi7S7u7tj9OjR+PnnnzFp0iQsXrz4aTbnqTwsIM3NzZ9oOXv37kVeXh7atWv3yH6FL16//fabwYvXsWPHlAvyJVl3cS+QBQUFT1Tz09JoNDAyMsKdO3fg6OgIAAYXbg8cOFBknrJ67J2dneHq6orTp08rwVQ4eHp6Kv1sbGzQv39/LF68GD/++CNWrlyJ69evA7i3n3v06IHPPvsM8fHxSEhIwOHDh0tVT3GeiW8RraoCAwMRGhqK3Nxc5UgAADp27IiwsDDk5OQgMDAQlpaWGDNmDN566y3Y29ujdu3amD17Nm7fvo0RI57sZ/fGjx+P//znP6hfvz68vb3xySefVPkPtqSlpaFv374YPnw4mjVrBmtra+zfvx+zZ89Gz549lX7Lly9Hq1at0L59eyxduhR79+7F119/DQAYPHgwIiMjERISgunTp+PatWsYN24chgwZ8thvk61Tpw62b9+OAQMGQKvVombNmpgwYQK6deuGBg0aID09HVu3blUCpyrw8fHB6tWr0axZM1y8eBEnTpwo0dHAL7/8gu7duxu8O3/Y8rVaLc6fP2/w3L1fs2bNsGTJEuTm5j7yaKCiZWdnIyUlBQCQnp6Ozz//HFlZWejRowe8vLzg7u6O6dOn4/3338eJEycwZ84cg/nL+rGfMWMG3njjDdja2iI4OBjZ2dnYv38/0tPTER4ejk8++QQuLi74xz/+ASMjIyxfvhx6vR52dnaIiYlBfn4+fH19YWFhge+//x7m5ubw8PB4qn1koExPLpGBM2fOCADx9vY2aD979qwAkIYNGyptd+7ckXHjxknNmjVFq9XK888/L3v37lWmF3cuU6Toxbzc3FwZP3682NjYiJ2dnYSHh8urr75apa8J3L17V9555x157rnnxNbWViwsLKRhw4YyZcoUuX37tojcuyYwf/58eeGFF0Sr1UqdOnXkxx9/NFjOoUOHJDAwUHQ6ndjb28vIkSPl5s2byvQH7xwplJCQIM2aNROtViuF/xJhYWFSr1490Wq14ujoKEOGDJHU1NTy2wkPkZqaKoGBgfLdd9/JwYMH5fTp0/LTTz+Js7OzDB8+XEREAgICpEmTJrJp0yY5ffq0rFu3TtavXy8iRa8JNG7cWFauXFnsuh58Lv373/8WBwcHiYmJkVOnTkliYqJ89tlnyl0tqamp4uDgoFwYPnHihHz77bcGF4Yf3N/jx4+Xjh07ls3OKUZISIgAUAZra2tp3bq1rFixQumzc+dOadq0qeh0OunQoYMsX77c4JrAox774q4JPHjNZdWqVfLgS+vSpUulRYsWYmZmJjVq1BB/f3/5+eefReTeTQ0tWrQQS0tLsbGxkc6dO8sff/yhLMvX11dsbGzE0tJS2rZtK5s3by7TfcYQoGoBgKxataqyy6hwJQnItLQ0GTZsmDg4OIhOp5MmTZrI2rVrRcTwRerUqVOi1WoN7qC634MhUFBQIHPnzpWGDRuKqampODo6SteuXQ1uPDh48KB06dJFLCwsxNraWjp06CBJSUkiUjkhQE+OvydA1YJGo8GqVav4HUhP4ZNPPsHmzZsr7UNvVDXxwjCRSri5uSEiIqKyy6AqhkcCREQqxiMBIiIVYwgQEakYQ4CISMUYAkREKsYQICJSMYYAEZGKMQSIiFSMIUBEpGIMASIiFft/A0FMvwfmYwoAAAAASUVORK5CYII=",
      "text/plain": [
       "<Figure size 400x300 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "class_names = [\"World\", \"Sports\", \"Business\", \"Sci/Tech\"]\n",
    "label_names = [class_names[label] for label in ds_sm['label']]\n",
    "\n",
    "plt.figure(figsize=(4, 3))\n",
    "sns.countplot(x=label_names, hue=label_names, palette=\"pastel\")\n",
    "plt.title(\"#Samples in Each Class\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2. Cohere Classify Endpoint"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The [Cohere Classify](https://docs.cohere.com/reference/classify) endpoint makes a prediction about which labels fit the specified text inputs best. To make a prediction, [Cohere Classify](https://docs.cohere.com/reference/classify) uses the provided examples of text + label pairs as a reference."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "import cohere\n",
    "from cohere.responses.classify import Example\n",
    "\n",
    "_ = load_dotenv(find_dotenv()) # read local .env file\n",
    "cohere_api_key = os.environ['COHERE_API_KEY']\n",
    "\n",
    "co = cohere.Client(cohere_api_key)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 2.1 Prepare X, y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1000, 1000)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = ds_sm['text']\n",
    "\n",
    "labels = {0: \"World\", 1: \"Sports\", 2: \"Business\", 3: \"Sci/Tech\"}\n",
    "y = [labels[x] for x in ds_sm['label']]\n",
    "\n",
    "len(X), len(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 2.2 Prepare Examples and Input for Cohere Classify Endpoint"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(905, 95)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.095, random_state=42)\n",
    "\n",
    "# Examples provide context to the model. Each example is a text string and its associated label/class. \n",
    "examples = [Example(text, label) for text, label in zip(X_train, y_train)]\n",
    "\n",
    "# Inputs represent a list of queries to be classified. The maximum is 96 inputs. We will use this classification to the assess the performance of the model.\n",
    "inputs = X_test\n",
    "\n",
    "len(examples), len(inputs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 2.3 Run Classification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "response = co.classify(\n",
    "  model=\"embed-english-v2.0\",\n",
    "  inputs=inputs,\n",
    "  examples=examples,\n",
    ")\n",
    "\n",
    "y_pred = [item.predictions[0] for item in response]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 2.4 Assess Performance"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The [scikit classification_report](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.classification_report.html) provides several key metrics for evaluating the performance of a classification model:\n",
    "\n",
    "- *Precision*: Precision is the ratio of correctly predicted positive observations to the total predicted positive observations. It's a measure of a classifier's exactness. For example, a precision of 0.90 for Business means that 90% of the instances predicted as Business were actually Business.\n",
    "- *Recall*: Recall (also known as sensitivity) is the ratio of correctly predicted positive observations to all observations in the actual class. It's a measure of a classifier's completeness. For example, a recall of 1.00 for World means that 100% of the actual World instances were correctly identified by the model.\n",
    "- *F1-Score*: The F1-score is the weighted average of Precision and Recall. It takes both false positives and false negatives into account. An F1-score reaches its best value at 1 (perfect precision and recall).\n",
    "- *Support*: The support is the number of actual occurrences of the class in the specified dataset. For example, there are 20 instances of the class Business in the dataset.\n",
    "\n",
    "- *Accuracy*: The accuracy of the model is the ratio of correctly predicted observations to the total observations. An accuracy of 0.95 means that 95% of all predictions were correct.\n",
    "- *Macro Avg*: Computes the metric independently for each class and then takes the average (hence treating all classes equally). For example, the macro avg precision of 0.94 means the average precision across all classes is 94%.\n",
    "- *Weighted Avg*: Takes the average of the metrics, weighted by the number of instances (support) of each class. This is useful if class imbalance is present in the dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 94.74%\n",
      "\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "    Business       0.90      0.90      0.90        20\n",
      "    Sci/Tech       0.96      0.92      0.94        24\n",
      "      Sports       1.00      0.96      0.98        28\n",
      "       World       0.92      1.00      0.96        23\n",
      "\n",
      "    accuracy                           0.95        95\n",
      "   macro avg       0.94      0.95      0.94        95\n",
      "weighted avg       0.95      0.95      0.95        95\n",
      "\n"
     ]
    }
   ],
   "source": [
    "accuracy = accuracy_score(y_test, y_pred)\n",
    "print(f\"Accuracy: {accuracy * 100:.2f}%\\n\")\n",
    "\n",
    "print(classification_report(y_test, y_pred))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "----"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}