classifier-linear-svm.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "1884b2d0",
   "metadata": {},
   "source": [
    "# Метод опорных векторов с линейным ядром (классификация)\n",
    "\n",
    "Используем реализацию классификатора на основе [линейного метода опорных векторов](https://scikit-learn.org/stable/modules/svm.html#) в `scikit-learn`: [LinearSVC](https://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVC.html#sklearn.svm.LinearSVC)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "37fa9e41-6253-43ec-83cd-237c03a66ccc",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "from sklearn.svm import LinearSVC\n",
    "from sklearn.model_selection import train_test_split\n",
    "# базовые метрики классификации\n",
    "from sklearn.metrics import accuracy_score, f1_score, auc, class_likelihood_ratios\n",
    "# ROC-curve, AUC\n",
    "from sklearn.metrics import roc_curve, roc_auc_score, RocCurveDisplay\n",
    "# Precision-recall\n",
    "from sklearn.metrics import precision_recall_curve, average_precision_score, PrecisionRecallDisplay\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Не показывать FutureWarnings\n",
    "import warnings\n",
    "warnings.simplefilter(action='ignore', category=FutureWarning)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90904ab0",
   "metadata": {},
   "source": [
    "## Импорт и подготовка данных"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b2f1b747",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv('./datasets/loanapp.csv')\n",
    "df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "24d520db",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df.dropna()\n",
    "df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ed4ae0f7-979b-4b9e-8c05-51b522f7e1f7",
   "metadata": {},
   "outputs": [],
   "source": [
    "y = df['approve']\n",
    "X = df.drop(columns='approve')\n",
    "# Разобьём выборку на обучающую и тестовую 80:20\n",
    "X_train,X_test,y_train,y_test = train_test_split(X,y,test_size=0.2,random_state=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b27f9bc",
   "metadata": {},
   "source": [
    "## Инициализация и обучение модели "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f14569ac-a76c-4a20-9d26-3a9602d77825",
   "metadata": {},
   "outputs": [],
   "source": [
    "# инициализируем и обучим классификатор\n",
    "clf = LinearSVC(penalty='l2', C=1.0)\n",
    "clf.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a8bc8a8",
   "metadata": {},
   "source": [
    "параметры/веса обученной модели"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ab833ea5",
   "metadata": {},
   "outputs": [],
   "source": [
    "# веса при признаках\n",
    "clf.coef_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "91cfaafd",
   "metadata": {},
   "outputs": [],
   "source": [
    "# константа w0\n",
    "clf.intercept_"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8d29c7c9",
   "metadata": {},
   "source": [
    "## Прогнозирование (на тестовой выборке)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1ec1e65b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# прогноз на тестовой выборке\n",
    "clf.predict(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2844fdcb",
   "metadata": {},
   "source": [
    "## Метрики качестве прогнозов на тестовой выборке"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d67977d7",
   "metadata": {},
   "outputs": [],
   "source": [
    "# доля правильных прогнозов на тестовой выборке\n",
    "clf.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3450a00a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# альтернативно: метод accuracy_score\n",
    "y_pred = clf.predict(X_test)\n",
    "accuracy_score(y_test, y_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b13654d",
   "metadata": {},
   "source": [
    "Метрика \n",
    "\n",
    "$$\n",
    "\tF1=\\frac{TP}{2TP+FP+FN}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c5d1f022",
   "metadata": {},
   "outputs": [],
   "source": [
    "f1_score(y_test, y_pred)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43e652ba",
   "metadata": {},
   "source": [
    "ROC-кривая и мера AUC"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e4d8947e",
   "metadata": {},
   "outputs": [],
   "source": [
    "RocCurveDisplay.from_estimator(clf, X_test, y_test)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3745f029",
   "metadata": {},
   "source": [
    "Кривая Precision-Recall"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1d4ebbd3",
   "metadata": {},
   "outputs": [],
   "source": [
    "PrecisionRecallDisplay.from_estimator(clf, X_test, y_test)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da39c084",
   "metadata": {},
   "source": [
    "## Сравнение нескольких классификаторов"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "568dde4d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Другой классификатор: SVM со штрафом l2 и C=0.5\n",
    "clf2 = LinearSVC(penalty='l2', C=0.5)\n",
    "clf2.fit(X_train, y_train)\n",
    "\n",
    "# Другой классификатор: SVM со штрафом l2 и C=2\n",
    "clf3 = LinearSVC(penalty='l2', C=2)\n",
    "clf3.fit(X_train, y_train)\n",
    "\n",
    "# Другой классификатор: SVM со штрафом l2 и C=1 и loss='hinge'\n",
    "clf4 = LinearSVC(penalty='l2', C=1, loss='hinge')\n",
    "clf4.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "59875af7",
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, axs = plt.subplots(nrows=1, ncols=2, figsize=(12, 8))\n",
    "\n",
    "RocCurveDisplay.from_estimator(clf, X_test, y_test, ax=axs[0], name='l2, C=1')\n",
    "RocCurveDisplay.from_estimator(clf2, X_test, y_test, ax=axs[0], name='l2, C=0.5')\n",
    "RocCurveDisplay.from_estimator(clf3, X_test, y_test, ax=axs[0], name='l2, C=2')\n",
    "RocCurveDisplay.from_estimator(clf4, X_test, y_test, ax=axs[0], name='l2, C=1, hinge')\n",
    "\n",
    "PrecisionRecallDisplay.from_estimator(clf, X_test, y_test, ax=axs[1], name='l2, C=1')\n",
    "PrecisionRecallDisplay.from_estimator(clf2, X_test, y_test, ax=axs[1], name='l2, C=0.5')\n",
    "PrecisionRecallDisplay.from_estimator(clf3, X_test, y_test, ax=axs[1], name='l2, C=2')\n",
    "PrecisionRecallDisplay.from_estimator(clf4, X_test, y_test, ax=axs[1], name='l2, C=1, hinge')\n",
    "\n",
    "axs[0].set_title('ROC-curves')\n",
    "axs[1].set_title('PR-curves')\n",
    "\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}