Univariate and Multivariate Charts and Plots Notebook .ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "3594d739",
   "metadata": {},
   "source": [
    "# Work-a-long Notebook for:\n",
    "## [Understand the Data With Univariate And Multivariate Charts and Plots in Python by Rashida Nasrin Sucky](https://towardsdatascience.com/understand-the-data-with-univariate-and-multivariate-charts-and-plots-in-python-3b9fcd68cd8) \n",
    "\n",
    "\n",
    "This notebook is designed to be used as you read through the article.  \n",
    "\n",
    "**Note:** This is not an exact one-to-one walkthrough.  You will find extra questions, extra python methods, and extra plots.  The goal is to help you better understand how to create questions about your data to discover correlations and patterns.  Also, this is NOT the only way to explore your data set.\n",
    "\n",
    "**Chart Context:** The [Heart dataset](https://www.kaggle.com/johnsmith88/heart-disease-dataset) is also part of the GitHub folder.  Be sure to explore the context provided with the dataset on Kaggle."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "547249a7",
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "import pandas as pd\n",
    "from statsmodels import api as sm\n",
    "import numpy as np\n",
    "\n",
    "df = pd.read_csv(\"heart.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0ab316bf",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Note:  make sure you provide the correct file path for your 'df' variable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "416d702d",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7df5409c",
   "metadata": {},
   "source": [
    "##### Question: (Answer to yourself)\n",
    "Have you looked at the context behind the data on kaggle? (link in top box)\n",
    "Do you know or have an understanding of each item being studied?\n",
    "If not, have you done some basic research yet?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3a92e3f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# The author of the article has already devised a few questions for us to work through.\n",
    "# As we continue, you will be asked questions about other items within the data.\n",
    "# This is to help you experience the question/answer phase, and help you better explore your data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e737e424",
   "metadata": {},
   "source": [
    "### A Quick Note!\n",
    "\n",
    "The data in this article has already been cleaned up. \n",
    "The exact methods have not be disclosed.  In this notebook, we cleaned the data in a way we felt worked best.  However, the final outcome is 302 records instead of 303.  Please be aware of this slight discrepency when checking your answers with the walkthrough.  Your values might be slightly smaller, but the exploration process will still be of value to you.\n",
    "\n",
    "Data cleaning includes removing duplicate records, values can be updated, formatting the index names, etc.  We will cover this topic in more depth in the next lesson.  In the next few blocks, you will see a sneak peek of the cleaning process.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "240819d6",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Print the column names  \n",
    "df.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d57db3a6",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Note the difference between your column names and the names from the article\n",
    "# Since the names are strings, let's use a string method to make them match\n",
    "\n",
    "df.columns = df.columns.str.capitalize()\n",
    "df.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d639618d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Renaming columns for easier comprehension\n",
    "df = df.rename(columns={\"Cp\": \"ChestPain\", \"Trestbps\": \"RestBP\", \"Restecg\": \"RestECG\", \"Exang\":\"ExAng\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "94860065",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Checking initial size of dataframe\n",
    "df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "982475f0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Checking for duplicates\n",
    "duplicate_rows_df = df[df.duplicated()]\n",
    "print(\"Number of duplicated rows: \", duplicate_rows_df.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e18cef84",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Dropping the duplicates then rechecking the shape\n",
    "df = df.drop_duplicates()\n",
    "df.shape\n",
    "\n",
    "# Note, our final count is 302.  \n",
    "# The final count in the article is 303.  \n",
    "# This small discrepency should not affect your values enough to prevent you from walking through the artcle."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2f5a2956",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Updating the values in \"Sex\" where 0 = \"Female\" and 1 = \"Male\"\n",
    "\n",
    "df = df.replace({\"Sex\": {0: \"Female\", 1: \"Male\"}})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e9f87cbc",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Updating the values in the \"Thal\" column to reflect what each value means.\n",
    "\n",
    "df = df.replace({\"Thal\": {1: \"Normal\", 2: \"Fixed\", 3: \"Reversable\", 0: \"Missing\"}})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cb2a5a26",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Updating the values in the \"ChestPain\" column to reflect what each number means\n",
    "\n",
    "df = df.replace({\"ChestPain\" : {0 : \"asymptomatic\", 1 : \"nonanginal\", 2 : \"nontypical\", 3 : \"typical\"}})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "273f20b6",
   "metadata": {},
   "outputs": [],
   "source": [
    "# testing replaced updates\n",
    "#df.head()\n",
    "\n",
    "\n",
    "# Here ends the data cleaning section"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e40dc922",
   "metadata": {},
   "source": [
    "### Solve Some Questions"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "31a3db20",
   "metadata": {},
   "source": [
    "### Question 1:  Find the population proportions with different types of blood disorders\n",
    "\n",
    "**Part A:** In this example we are going to use the \"Thal\" column.\n",
    "Follow along with the article for syntax and explanations of the output.\n",
    "\n",
    "*Remember:* with duplicated rows dropped, our final record count is 302 and the author's is 303."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7062cb56",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Definitions and Question:\n",
    "\n",
    "# Define 'value_counts()':\n",
    "\n",
    "# Why is this method used as part of initalizing a new varible?\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3576aab9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define a variable for the value counts of the Thal column.\n",
    "x = df.Thal.value_counts()\n",
    "x\n",
    "\n",
    "# Note: In the data set, some values were passed 0 rather than left missing.  \n",
    "# These values have been replaced with \"Missing\".\n",
    "# Having this already filled in, will not affect your walkthrough, at this point"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0e605fc2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Calculate the population proportion here:\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c750b408",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Turn those proportions into percentages:\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d7b4c762",
   "metadata": {},
   "source": [
    "**Part B:**  Let's examine the proportions of other populations within this dataset.\n",
    "\n",
    "Let's look at \"ExAng\" or, as the provided context informs us, exercise induced angina.\n",
    "\n",
    "We are going to follow the steps provided with the \"Thal\" values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8b76d6d7",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define a variable for the value counts of the Exang column.\n",
    "\n",
    "# y = \n",
    "# y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "efdfea87",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Calculate the population proportions:\n",
    "\n",
    "# Look at the context to see what 1 and 0 mean."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6de7044b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Turn those propotions into percentages:\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "704f6bfa",
   "metadata": {},
   "source": [
    "**Part C:**  Select one of the other populations to examine."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "05e49218",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define a variable for the value counts of the your column.\n",
    "\n",
    "# z = \n",
    "# z"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "86b86095",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Calculate the population proportions:\n",
    "\n",
    "# Does the context provide you with any information about the values?."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6679c499",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Turn those propotions into percentages:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa68b8ef",
   "metadata": {},
   "source": [
    "### Question 2: Find the minimum, maximum, average, and standard deviation of Cholesterol data.\n",
    "\n",
    "Remember: Our total is 302, not 303."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "47eb75f3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# using only the Cholesterol column in the dataset, apply the describe function.\n",
    "\n",
    "df[\"Chol\"].describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4dd6ad23",
   "metadata": {},
   "source": [
    "### Question 3: Make a plot of the distribution of the Cholesterol data.\n",
    "\n",
    "**Part A:** Plot the Cholestorol data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ad73996b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot the distribution here:\n",
    "\n",
    "sns.displot(df.Chol.dropna())\n",
    "\n",
    "# Why is this a \"histplot\" and not a \"distplot\"?  \n",
    "# According to the seaborn (sns) documentation, \n",
    "#  the \"distplot\" is a depricated function and will be removed in a future version.\n",
    "# So let's explore other options in this learning space.\n",
    "\n",
    "# Could also try to use sns.histplot(df.Chol.dropna())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "99adcd81",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Do you see any outliers?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fe25d610",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Want the histogram with the standard deviation curve on the same plot? \n",
    "\n",
    "# Here is they syntax:\n",
    "sns.histplot(data=df, x=\"Chol\", kde=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "589eaae8",
   "metadata": {},
   "source": [
    "[Documentation on the histplot](https://seaborn.pydata.org/generated/seaborn.histplot.html) to better understand the syntax."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb864fc4",
   "metadata": {},
   "source": [
    "**Part B:** Describe and Plot the Resting Heart Rate values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "600c0159",
   "metadata": {},
   "outputs": [],
   "source": [
    "# code the describe() for RestBP\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e6abc7d1",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's plot The resting heart rate, \"RestBP\".  \n",
    "#You can choose to plot using \"histplot\" or \"displot\".  Your choice.\n",
    "sns.displot(df.RestBP.dropna())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2f5f5169",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Any outliers?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ae44e95",
   "metadata": {},
   "source": [
    "**Part C:** Describe and Plot the values you selected in Question 1 part c."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5518859b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot Part C here: \n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "616f8dbd",
   "metadata": {},
   "source": [
    "### Question 4: Find the mean of the RestBP and calculate the population proportion of the people who have the higher RestBP than the mean RestBP."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9c9cf569",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Calculate the mean of the RestBp and print it\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4ba8a136",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Question: Is the .dropna() siginificant or necessary for calculating the mean_rbp?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5181aed0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Calculate population proportion of people who have higher RestBP than the average\n",
    "\n",
    "\n",
    "# Questions: \n",
    "\n",
    "# 1. Translate this line of code in plain English?\n",
    "\n",
    "# 2.  Break this line and examine your results.  What did you change?  What happened?\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "239b8b9e",
   "metadata": {},
   "source": [
    "### Question 5: Plot the Cholesterol data against the age group"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "505ad95e",
   "metadata": {},
   "outputs": [],
   "source": [
    "df[\"agegrp\"]=pd.cut(df.Age, [29,40,50,60,70,80])\n",
    "plt.figure(figsize=(12,5))\n",
    "sns.boxplot(x = \"agegrp\", y = \"Chol\", data=df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b710447c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Questions for you to answer:\n",
    "\n",
    "# Looking at the code that was used to create the box plot, \n",
    "#   explain what the following code means in your own words:\n",
    "\n",
    "# 1. pandas.cut():\n",
    "\n",
    "# 2. Looking at the cut() method signature, what is a \"bin\": \n",
    "\n",
    "# 3. plt.figure():\n",
    "\n",
    "# 3 A - in the code above, what do the numbers in figsize mean?  What happens if you change them?\n",
    "\n",
    "# 4. sns.boxplot(x = ??, y = ??, data= ??):\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "02f4b76f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Combine sex and cholesterol data here:\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "# Why did the author run this: df[\"Sex1\"] = df.Sex.replace({1: \"Male\", 0: \"Female\"})\n",
    "\n",
    "# What does that line mean, in your own words:\n",
    "\n",
    "\n",
    "# There is a new argument in the boxplot function.  Look up what \"hue\" means."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2bcd4065",
   "metadata": {},
   "outputs": [],
   "source": [
    "# How does the sns.boxplot(x = \"Sex1\", y = \"Age\", data=df) differ from the one in the cell above?\n",
    "\n",
    "# What information does this provide you?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "54a126dc",
   "metadata": {},
   "source": [
    "### Question 6:  Make a chart to show the number of people having each type of chest pain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "63cff4a2",
   "metadata": {},
   "outputs": [],
   "source": [
    "df.groupby(\"agegrp\")[\"ChestPain\"].value_counts().unstack()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bf4b4ed9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run the following line of code:\n",
    "\n",
    "#df.groupby(\"agegrp\")[\"ChestPain\"].value_counts()\n",
    "\n",
    "# Question:  \n",
    "# 1. How does the output vary between these two lines?  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7eb8c233",
   "metadata": {},
   "source": [
    "### Question 7: Add on to your chart, but segregate by gender."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d516057d",
   "metadata": {},
   "outputs": [],
   "source": [
    "dx = df.dropna().groupby([\"agegrp\", \"Sex\"])[\"ChestPain\"].value_counts().unstack()\n",
    "print(dx)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a437830",
   "metadata": {},
   "source": [
    "### Question 8:  Present population proportion for each type of chest pain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7bb8ec74",
   "metadata": {},
   "outputs": [],
   "source": [
    "dx = dx.apply(lambda x: x/x.sum(), axis=1)\n",
    "print(dx)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "362afca8",
   "metadata": {},
   "source": [
    "Note on your output, as we made this notebook, we discovered that the ariticle dataset had been cleaned prior to our usage.  The methods used were not disclosed, so we cleaned ourselves.  Which could explain the discrepencies in this chart."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "77718b7e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Questions: In your own words:\n",
    "\n",
    "# 1. What does the apply() function do?\n",
    "\n",
    "\n",
    "# 2. What is lambda? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6eb6f8e0",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}