CIF_pull_dataGC.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-m-IXs4afdm4"
   },
   "source": [
    "# CIFTools Google Colab Runtime\n",
    "\n",
    "The following will allow you to pull data from several Federal data sources and filter them to your catchment area or other geographic area of interest.\n",
    "\n",
    "To begin, you will need to install or upgrade some modules on your Google Colab runtime. This only needs to be done the first time you run this on each visit. \n",
    "\n",
    "*After successful completion you may want to clear the terminal to reduce clutter*."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ER88kcaef3Ef"
   },
   "source": [
    "Next, you will need to upload the `ciftools_v2-main.zip`.    \n",
    "\n",
    "Then executing `unzip`, all the files for the CIFTools will be set-up and ready-to-use in the `ciftools_v2-main` folder.    \n",
    "Within the `ciftools_v2-main`, there are not only the `CIFTools.py` module and all the catchment area csv files within the `catment_area` folder."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!git clone https://github.com/leeparkuky/ciftools_v2.git"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we will change the working directory for both terminal and python kernel within the notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import os\n",
    "os.chdir(os.path.join(os.getcwd(), 'ciftools_v2'))\n",
    "%pip install -r requirements.txt\n",
    "from utils import write_bash_script"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Option 1: If you find a catchment area file under `catchment_area` folder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bash_script_kwargs = {\n",
    "    \"bash_file_name\" : 'googleColabMain.sh', #the name of a bash file to run\n",
    "    \"catchment_area_name\": \"Markey Cancer Center\", # the name of the catchment area name\n",
    "    \"ca_file_path\": \"uky_ca.csv\",\n",
    "    \"query_level\" : ['county','tract'],\n",
    "    \"acs_year\"    : 2019,\n",
    "    \"download_file_type\": ['pickle','csv', 'excel'],\n",
    "    \"census_api_key\": 'f1a4c4de1f35fe90fc1ceb60fd97b39c9a96e436',\n",
    "    \"generate_zip_file\" : True,\n",
    "    \"install_packages\" : False # We already installed required packages above\n",
    "#     \"socrata_user_name\": \"...\", # provide socrata user name and password if you have\n",
    "#     \"socrata_password\": \"...\"\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'all_catchment_data.zip'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "write_bash_script(**bash_script_kwargs)    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pickle"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "!bash googleColabMain.sh"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Option 2: When you want to upload a custom catchment area file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from utils import import_custom_ca_file\n",
    "\n",
    "ca_file_path = import_custom_ca_file()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bash_script_kwargs = {\n",
    "    \"bash_file_name\" : 'googleColabMain.sh', #the name of a bash file to run\n",
    "    \"catchment_area_name\": \"Appalachian\", # the name of the catchment area name\n",
    "    \"ca_file_path\": ca_file_path,\n",
    "    \"query_level\" : ['county','tract'],\n",
    "    \"acs_year\"    : 2019,\n",
    "    \"download_file_type\": ['pickle','csv', 'excel'],\n",
    "    \"census_api_key\": 'f1a4c4de1f35fe90fc1ceb60fd97b39c9a96e436',\n",
    "    \"generate_zip_file\" : True,\n",
    "    \"install_packages\" : False # We already installed required packages above\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "write_bash_script(**bash_script_kwargs)    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!bash googleColabMain.sh"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Downloading data zip file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from utils import download_zip_files\n",
    "\n",
    "download_zip_files()"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}