build_deploy.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Appendix B - Building and Deploying the App"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* [Part 1 - The Webservice](#webservice)\n",
    "* [Part 2 - The Shiny User Interface](#shiny)\n",
    "* [Part 3 - The Static Web Site](#squarespace)\n",
    "* [References](#references)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"webservice\"></a>\n",
    "## Part 1 - The Webservice"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The code for the webservice can be found in the ./webservice subdirectory. This is a copy of the code running in production. The reference origin git repo is *not* in this repository. It is hosted on [Heroku](https://www.heroku.com/). \n",
    "\n",
    "### How the Webservice Works\n",
    "\n",
    "The webservice is a minimal [Flask](http://flask.pocoo.org/) application used to provide prediction probabilities for a given candidate. There is one useful entry point: `/predict` that expects the user's admissionTest, AP, etc on the query string. It will return a JSON file consisting of the probabilities of getting into each college.\n",
    "\n",
    "Sample input:\n",
    "\n",
    "```\n",
    "http://boiling-forest-8250.herokuapp.com/predict?admissionstest=0.926899206&AP=7&averageAP=1.06733864&SATsubject=0.324271565&GPA=-0.187109979&schooltype=0&intendedgradyear=2017&female=1&MinorityRace=0&international=0&sports=0&earlyAppl=0&alumni=0&outofstate=0&acceptrate=0.151&size=6621&public=0&finAidPct=0&instatePct=0\n",
    "```\n",
    "\n",
    "Sample output:\n",
    "```\n",
    "{\n",
    "  \"preds\": [\n",
    "    {\n",
    "      \"college\": \"Princeton\",\n",
    "      \"prob\": 0.26166666666666666\n",
    "    },\n",
    "    {\n",
    "      \"college\": \"Harvard\",\n",
    "      \"prob\": 0.23999999999999999\n",
    "    },\n",
    "    {\n",
    "      \"college\": \"Yale\",\n",
    "      \"prob\": 0.23999999999999999\n",
    "    },\n",
    "    ...\n",
    " ]\n",
    "}\n",
    "```\n",
    "\n",
    "### Webservice startup\n",
    "\n",
    "Upon getting the first `/predict` request, the web service will perform the same logic as the classification iPython notebook. It loads the normalized college data, imputes missing values, and runs Scikit-Learn's Random Forest classification. The resulting classifier is kept in memory as a Python global variable to service subsequent prediction requests. There is no locking at the present time. \n",
    "\n",
    "### Webservice Dependencies - OpenShift\n",
    "\n",
    "We started using OpenShift. The free account worked fine at first and then had consistent and terrible performance problems a few days before the project was due. With two days remaining, we scrambled and moved from OpenShift to Heroku. This section serves as reference (and consider it a warning to not use OpenShit ever again).\n",
    "\n",
    "Since the webservice is running the full Pandas and Scikit-Learn stacks, these had to installed on the OpenShift cartridge. Here's what was done:\n",
    "\n",
    "1. Create an OpenShift account\n",
    "1. Install the [client tools](https://developers.openshift.com/en/managing-client-tools.html). This will install `rhc`, the necessary local command line tool for managing OpenShift apps.\n",
    "1. Use the Flask Quickstart template ([details](https://developers.openshift.com/en/python-flask.html))\n",
    "\n",
    "    ```\n",
    "    rhc app create myflaskapp python-2.7 --from-code=https://github.com/openshift-quickstart/flask-base.git\n",
    "    ```\n",
    "\n",
    "1. This will create a local myflaskapp git repository. Go into this repository: `cd myflaskapp`\n",
    "1. SSH into the app and install the dependent packages:\n",
    "\n",
    "    ```\n",
    "    rhc ssh myflaskapp\n",
    "    source ~/python/virtenv/activate\n",
    "    pip install numpy\n",
    "    ```\n",
    "\n",
    "    The `pip install` has to be repeated for `scipy, pandas` and `scikit-learn`. This takes a while as it is compiled     locally on the OpenShift instance and may not have optimal performance.\n",
    "\n",
    "1. After all the packages have been installed, take the output of `pip freeze` and update the `requirements.txt` in the *local* repository.\n",
    "1. At this point, you can grab the appropriate files from `./webservice` directory, notably: `TIdatabase.py, collegelist.csv, collegedata_normalized.csv, flaskapp.py`.\n",
    "\n",
    "\n",
    "\n",
    "#### DevOps Notes\n",
    "\n",
    "To see the logs, use `rhc tail -o '-n 100' mypythonapp`\n",
    "\n",
    "Common rhc commands can be found [here](https://developers.openshift.com/en/managing-common-rhc-commands.html)\n",
    "\n",
    "### Webservice Dependencies - Heroku\n",
    "\n",
    "Heroku was easier to configure since there are buildpacks available that contain the entire Condas stack with all the Scipy, Numpy, Scikit-learn dependencies. There were still numerous gotchas, mainly related to finding the right\n",
    "combination of scipy, numpy and scikit-learn versions that would all play nicely together.\n",
    "\n",
    "Heroku has a [nice walkthrough](https://devcenter.heroku.com/articles/getting-started-with-python#introduction) about setting up a Python app in minutes. I mostly followed that, with the following changes:\n",
    "\n",
    "Add the Conda buildpack:\n",
    "```\n",
    "heroku config:add BUILDPACK_URL=https://github.com/kennethreitz/conda-buildpack.git\n",
    "```\n",
    "\n",
    "This buildpack has a broken scipy, so it was obtained from:\n",
    "```\n",
    "heroku buildpacks:set https://github.com/thenovices/heroku-buildpack-scipy\n",
    "```\n",
    "\n",
    "This version scipy is broken with the latest scikit-learn, so I had to downgrade scikit-learn. Here is our final requirements.txt file:\n",
    "```\n",
    "gunicorn==19.3.0\n",
    "psycopg2==2.6\n",
    "SQLAlchemy==1.0.4\n",
    "whitenoise==1.0.6\n",
    "Flask==0.10.1\n",
    "pandas==0.17.1\n",
    "numpy==1.9.1\n",
    "scipy==0.15.1\n",
    "scikit-learn==0.16.1\n",
    "nose==1.3.7\n",
    "```\n",
    "\n",
    "and our Procfile\n",
    "```\n",
    "web: gunicorn flaskapp:app --log-file=-\n",
    "```\n",
    "\n",
    "After the pain and suffering with OpenShift's free account, we went with the Heroku \\$7/mo hobbyist dyno with the hope that the app would not go down again.\n",
    "\n",
    "### Programming Notes\n",
    "\n",
    "The real work is done in `flaskapp.py`.\n",
    "\n",
    "Logging is off by default. To log errors from your app, use:\n",
    "\n",
    "```\n",
    "import logging\n",
    "\n",
    "logging.basicConfig(level=logging.DEBUG,format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')\n",
    "```\n",
    "\n",
    "We simply used global variables to stored information. It is also possible to use the [appcontext](http://flask.pocoo.org/docs/0.10/appcontext/). \n",
    "\n",
    "Note that the app can be tested locally very easily. From a local shell use: `python flaskapp.py`. It will say which address / port it is listening on when starting up.\n",
    "\n",
    "### Consuming the Webservice from R\n",
    "\n",
    "Sample code to consume the webservice can be found in `rclient.R`. This simulates how the production Shiny app can invoke the webservice. An R data.frame is created with the normalized user inputted values. This is used to populate the query string of the webservice. Note that the webservice ignores the last five variables, which are specific to a given college, since probabilities for *all* colleges are returned.\n",
    "\n",
    "The returned JSON is easily parsed into an R data.frame for presentation to the user or further manipulation. Here is a snippet:\n",
    "\n",
    "```\n",
    "# create query string\n",
    "qs = paste0(colnames(pred),\"=\",pred[1,],collapse=\"&\")\n",
    "server = \"http://127.0.0.1:5000/predict\"\n",
    "server = \"http://boiling-forest-8250.herokuapp.com/predict\"\n",
    "\n",
    "URL = paste0(server,\"?\",qs)\n",
    "\n",
    "js  = fromJSON(URL)\n",
    "df = js$preds\n",
    "df$college = as.factor(df$college)\n",
    "summary(df)\n",
    "\n",
    "```\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Webservice Code\n",
    "\n",
    "(This is not in a code cell because it is not meant to be executed)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "```\n",
    "from flask import Flask\n",
    "from flask import jsonify, request\n",
    "\n",
    "import os\n",
    "import pandas as pd\n",
    "from sklearn.preprocessing import Imputer\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "import numpy as np\n",
    "import logging\n",
    "\n",
    "import TIdatabase as ti\n",
    "\n",
    "app = Flask(__name__)\n",
    "\n",
    "clf = None\n",
    "logging.basicConfig(level=logging.DEBUG,format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')\n",
    " \n",
    "\n",
    "ws_cols = [\"admissionstest\",\"AP\",\"averageAP\",\"SATsubject\",\"GPA\",\"schooltype\",\n",
    "                  \"female\",\"MinorityRace\",\"international\",\"sports\",\n",
    "                  \"earlyAppl\",\"alumni\",\"outofstate\"]\n",
    "college_cols = [\"acceptrate\",\"size\",\"public\"]\n",
    "predictor_cols = ws_cols + college_cols\n",
    "\n",
    "cols_to_drop = ['classrank', 'canAfford', 'firstinfamily', 'artist', 'workexp', 'visited', 'acceptProb',\n",
    "                'addInfo','intendedgradyear']\n",
    "NUM_ESTIMATORS = 1000\n",
    "\n",
    "colleges = ti.College()\n",
    "\n",
    "def load_classifier():\n",
    "    global clf\n",
    "    df = pd.read_csv(os.path.join(os.path.dirname(__file__),\"collegedata_normalized.csv\"), index_col=0)\n",
    "    dfr = df.drop(cols_to_drop,axis=1)\n",
    "    dfr = dfr[pd.notnull(df[\"acceptStatus\"])]\n",
    "    dfpredict = dfr[predictor_cols]\n",
    "    dfresponse = dfr[\"acceptStatus\"]\n",
    "    imp = Imputer(missing_values=\"NaN\", strategy=\"median\", axis=1)\n",
    "    imp.fit(dfpredict)\n",
    "    X = imp.transform(dfpredict)\n",
    "    y = dfresponse\n",
    "    clf = RandomForestClassifier(n_estimators=NUM_ESTIMATORS, criterion=\"gini\")\n",
    "    clf.fit(X,y)\n",
    "    return clf\n",
    "\n",
    "def genPredictionList(vals):\n",
    "    \"\"\"\n",
    "    vals (coming from the request arguments) is a list of tuples [('name1','val1'),('name2','val2')...]\n",
    "    \"\"\"\n",
    "    global ws_cols\n",
    "    global clf\n",
    "    global colleges\n",
    "    X = pd.Series(dict((name, float(val)) for name, val in vals))\n",
    "    if clf is None: load_classifier()\n",
    "    preds = []\n",
    "    for i, row in colleges.df.iterrows():\n",
    "        X[college_cols] = row[college_cols]\n",
    "        y = clf.predict_proba(X[predictor_cols])[0][1]\n",
    "        p = {'college':row.collegeID, 'prob':y}\n",
    "        preds.append(p)\n",
    "    return preds\n",
    "    #e.g.  [{'college':'harvard', 'prob':y}, {'college':'yale', 'prob':0.25}, {'college':'brown', 'prob':0.89}]\n",
    "\n",
    "@app.route('/')\n",
    "def hello_world():\n",
    "    return \"Welcome to the Team Ivy Web Service\"\n",
    "\n",
    "@app.route(\"/predict\")\n",
    "def predict():\n",
    "    preds = genPredictionList(request.args.iteritems())\n",
    "    return jsonify(preds = preds)\n",
    "\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    app.run(debug=True)\n",
    "\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"shiny\"></a>\n",
    "## Part 2 - The Shiny Web Application\n",
    "\n",
    "[Shiny](http://shiny.rstudio.com/) is a web application framework for R. It allows rapid development of reactive web applications. In this project, Shiny is used to implement all user interaction including plots and charts.\n",
    "\n",
    "The Shiny app is hosted at http://www.shinyapps.io/\n",
    "\n",
    "We initially attempted to implement RandomForests in R, but the results were not consistent with the Python classification code. It was at that time that we decided to implement the Python webservice. \n",
    "\n",
    "The Shiny app consists of three files:\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`global.R` is common code for both the client and server. When doing the classification, we saved the normalization\n",
    "means and standard deviations so we could normalize user input precisely the same way. We load in those normalization values upon app startup.\n",
    "\n",
    "```\n",
    "## shape data for MST\n",
    "library(dplyr)\n",
    "load('model.RData')\n",
    "act2sat<-data.frame(sat=list(1600, 1560, 1510, 1460, 1420, 1380, 1340, 1300, 1260, 1220, 1190, 1150, 1110, 1070, 1030, 990, 950, 910, 870,\n",
    "                         830, 790, 740, 690, 640, 590, 530),act=list(36, 35, 34, 33, 32, 31, 30, 29, 28, 27, 26, 25, 24, 23, 22, 21, 20, 19,\n",
    "                                                                  18, 17, 16, 15, 14, 13, 12, 11))\n",
    "\n",
    "datanormed<-read.csv(\"collegedata_normalized.csv\")\n",
    "datanormed$SATsubject<-as.numeric(datanormed$SATsubject)\n",
    "\n",
    "drops <- c(\"X\",\"studentID\",\"classrank\",\"GPA_w\",\"program\",\"intendedgradyear\",\"addInfo\",\"canAfford\",\n",
    "           \"MinorityGender\",\"firstinfamily\",\"artist\",\"workexp\",\"collegeID\",\"visited\")\n",
    "dataunnormed$female[dataunnormed$female<1]<-0\n",
    "dataunnormed$MinorityRace[dataunnormed$MinorityRace<1]<-0\n",
    "dataunnormed$schooltype[dataunnormed$schooltype<1]<-0\n",
    "dataunnormed$earlyAppl[dataunnormed$earlyAppl<1]<-0\n",
    "dataunnormed$outofstate[dataunnormed$outofstate<1]<-0\n",
    "dataunnormed$alumni[dataunnormed$alumnil<1]<-0\n",
    "dataunnormed$international[dataunnormed$international<1]<-0\n",
    "dataunnormed$sports[dataunnormed$sports<1]<-0\n",
    "dataunnormed<-plyr::rename(dataunnormed,c(\"admissionstest\"= \"ACT/SAT score\",\n",
    "                                          \"GPA\"= \"GPA\",\n",
    "                                          \"averageAP\"= \"average AP score\",\n",
    "                                          \"AP\"= \"# AP exams taken\",\n",
    "                                          \"SATsubject\"= \"# SAT subject tests taken\",\n",
    "                                          \"female\"= \"female indicator\",\n",
    "                                          \"schooltype\"= \"private high-school indicator\",\n",
    "                                          \"MinorityRace\"= \"minority indicator\",\n",
    "                                          \"earlyAppl\"= \"early application\",\n",
    "                                          \"outofstate\"= \"out of state indicator\",\n",
    "                                          \"alumni\"= \"alumni indicator\",\n",
    "                                          \"international\"= \"international indicator\",\n",
    "                                          \"sports\"= \"varsity sports indicator\"))\n",
    "subdata<-dataunnormed[,1:27]\n",
    "subdata<-subdata[,!(names(subdata) %in% drops)]\n",
    "\n",
    "\n",
    "normedmeans<-read.csv(\"normalize_means.csv\")\n",
    "normedstds<-read.csv(\"normalize_stds.csv\")\n",
    "\n",
    "better_labels<-c(\"varsity sports indicator\",\n",
    "                 \"international indicator\",\n",
    "                 \"alumni indicator\",\n",
    "                 \"public university indicator\",\n",
    "                 \"out of state indicator\",\n",
    "                 \"early application\",\n",
    "                 \"minority indicator\",\n",
    "                 \"private high-school indicator\",\n",
    "                 \"female indicator\",\n",
    "                 \"# SAT subject tests taken\",\n",
    "                 \"# AP exams taken\",\n",
    "                 \"college size\",\n",
    "                 \"average AP score\",\n",
    "                 \"GPA\",\n",
    "                 \"ACT/SAT score\")\n",
    "\n",
    "```\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`ui.R` contains the user interface\n",
    "```\n",
    "\n",
    "library(dplyr)\n",
    "library(shiny)\n",
    "library(shinyBS)\n",
    "require(ggplot2)\n",
    "require(reshape2)\n",
    "require(plyr)\n",
    "library(randomForest)\n",
    "library(ggvis)\n",
    "\n",
    "\n",
    "\n",
    "\n",
    "shinyUI(fluidPage(\n",
    "  titlePanel(\"Chance me.\"),\n",
    "  fluidRow(column(12,align=\"center\",\n",
    "\n",
    "           wellPanel(\n",
    "              sliderInput(\"sat\", \"SAT composite score\",\n",
    "                         0, 2400, 1200, step = 10),\n",
    "              sliderInput(\"act\", \"ACT composite score\",\n",
    "                          0, 36, 18, step = 1),\n",
    "              sliderInput(\"gpa\", \"GPA (unweighted)\", 0,4, 2, step = .1),\n",
    "              sliderInput(\"apnum\", \"Number of AP exams taken\", 0,10, 5, step = 1),\n",
    "              sliderInput(\"apave\", \"Average AP score\",\n",
    "                         0, 5, 2.5, step = .1),\n",
    "              sliderInput(\"sat2ave\", \"Number of SAT Subject tests taken\",\n",
    "                          0, 10, 5, step = 1)\n",
    "\n",
    "  )\n",
    ",column(4,selectInput(\"hs\",\"What type of high school did you attend?\",\n",
    "                        c(\"Public\"=\"0\",\"Private\"=\"1\",\"Parochial\"=\"2\",\"Homeschool\"=\"3\")),\n",
    "           radioButtons(\"gender\",\"What gender do you identify as?\",\n",
    "                        c(\"Female\"=\"1\",\"Male\"=\"-1\",\"Other\"=\"0\")),\n",
    "           selectInput(\"race\",\"What ethnicity do you identify as?\",\n",
    "                       c(\"African American/Black\"=\"1\",\"Hispanic/Latino\"=\"2\",\n",
    "                         \"Asian\"=\"0\",\"Middle Eastern\"=\"-1\",\"Pacific Islander\"=\"3\",\n",
    "                         \"Native American\"=\"4\",\"White\"=\"-2\",\"Other\"=\"5\"))\n",
    "           \n",
    "        \n",
    "        \n",
    "),\n",
    "column(4, radioButtons(\"international\",\"Are you a foreign national?\",\n",
    "                       c(\"No\"=0,\"Yes\"=1)),\n",
    "       radioButtons(\"firstinfamily\",\"Are you the first in your family to attend university?\",\n",
    "                    c(\"No\"=0,\"Yes\"=1)),\n",
    "       \n",
    "       radioButtons(\"sports\",\"Do you play varsity athletics?\",\n",
    "                    c(\"No\"=0,\"Yes\"=1))                   ),\n",
    "       column(4, radioButtons(\"alum\",\"Are you a legacy at this school?\",\n",
    "                              c(\"No\"=0,\"Yes\"=1)),\n",
    "              radioButtons(\"out\",\"Are you applying from out of state?\",\n",
    "                           c(\"No\"=0,\"Yes\"=1)),\n",
    "              radioButtons(\"early\",\"Are you applying early?\",\n",
    "                           c(\"No\"=0,\"Yes\"=1))))),\n",
    "fluidRow(    column(12,align=\"center\",selectInput(\"college\", \"What college are you applying to?\",\n",
    "                    c(\"\",as.character(sort(unique(dataunnormed$name))))),\n",
    "              h2(uiOutput(\"headerText\")),\n",
    "              bsAlert(\"alert\"),\n",
    "              br(),\n",
    "              h4(uiOutput(\"importancehelper\")),\n",
    "              plotOutput(\"importance\"),\n",
    "              br(),\n",
    "              h4(uiOutput(\"scatterhelper\")),\n",
    "                conditionalPanel(condition=\"input.college!=''\",wellPanel(\n",
    "                  selectInput(\"xvar\", \"X-axis variable\", better_labels, selected = \"private high-school indicator\"),\n",
    "                  selectInput(\"yvar\", \"Y-axis variable\", better_labels, selected = \"GPA\"),\n",
    "                  tags$small(paste0(\n",
    "                    \"Choose different input variables to investigate how aspects of an application are related. For example, you might ask: Do private school students have better GPAs than public school students?\"))),\n",
    "              ggvisOutput(\"plot1\")),\n",
    "              \n",
    "              br()))))\n",
    "\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally `server.R` is the user interface back end that also invokes the webservice\n",
    "```\n",
    "library(shiny)\n",
    "library(shinyBS)\n",
    "require(ggplot2)\n",
    "require(reshape2)\n",
    "require(plyr)\n",
    "require(curl)\n",
    "library(randomForest)\n",
    "library(ggvis)\n",
    "library(jsonlite)\n",
    "\n",
    "\n",
    "# Define server logic\n",
    "shinyServer(function(input, output, session) {\n",
    "\n",
    " \n",
    " \n",
    " observeEvent(input$goButton, {gotime<-1\n",
    "  })\n",
    "  \n",
    " \n",
    "  \n",
    "  # get text for header\n",
    "  output$headerText <- renderUI({\n",
    "    \n",
    "    if (input$college!=\"\"){\n",
    "      \n",
    "    #translate SAT and ACT to combined score\n",
    "    if (as.numeric(input$act)==0 && as.numeric(input$sat)!=0)\n",
    "    {at = as.numeric(input$sat)}\n",
    "    else if (as.numeric(input$sat)==0 && as.numeric(input$act)!=0)\n",
    "    {\n",
    "      #convert act to sat\n",
    "      at = act2sat$sat[act2sat$act==as.numeric(input$act)]}\n",
    "    else\n",
    "    {at = 0}\n",
    "    #normalize against test data\n",
    "    gpa_normed<-(as.numeric(input$gpa) - normedmeans$GPA)/normedstds$GPA\n",
    "    at_normed<-(at - normedmeans$admissionstest)/normedstds$admissionstest\n",
    "    apave_normed<-(as.numeric(input$apave) - normedmeans$averageAP)/normedstds$averageAP\n",
    "    #sat2ave_normed<-(as.numeric(input$sat2ave) - normedmeans$SATsubject)/normedstds$SATsubject\n",
    "    \n",
    "    \n",
    "    #process values into boolean\n",
    "    if (as.integer(input$race)>0)\n",
    "      {race = 1}\n",
    "    else {race=0}\n",
    "    if (as.integer(input$hs)>0)\n",
    "    {hs = 1}\n",
    "    else {hs=0}\n",
    "    if (as.integer(input$gender)>0)\n",
    "    {fem = 1}\n",
    "    else {fem=0}\n",
    "    if (input$apnum==0)\n",
    "    {apave_normed=0}\n",
    "    \n",
    "    \n",
    "    pred = data.frame(admissionstest=numeric(0),\n",
    "                      AP=numeric(0),\n",
    "                      averageAP=numeric(0),\n",
    "                      SATsubject=numeric(0),\n",
    "                      GPA=numeric(0),\n",
    "                      schooltype=numeric(0),\n",
    "                      female=numeric(0),\n",
    "                      MinorityRace=numeric(0),\n",
    "                      international=numeric(0),\n",
    "                      sports=numeric(0),\n",
    "                      earlyAppl=numeric(0),\n",
    "                      alumni=numeric(0),\n",
    "                      outofstate=numeric(0),\n",
    "                      acceptrate=numeric(0),\n",
    "                      size=numeric(0),\n",
    "                      public=numeric(0))\n",
    "\n",
    "\n",
    "    pred[1,] = list(at_normed,  as.numeric(input$apnum), apave_normed,   input$sat2ave,\n",
    "                    gpa_normed,   hs,   fem,\n",
    "                    race,   as.numeric(input$international),   as.numeric(input$sports),   as.numeric(input$early),\n",
    "                    as.numeric(input$alum),   as.numeric(input$out),   \n",
    "                    datanormed$acceptrate[datanormed$name==input$college][1],   \n",
    "                    datanormed$size[datanormed$name==input$college][1],\n",
    "                    datanormed$public[datanormed$name==input$college][1])\n",
    "    \n",
    "    \n",
    "    # create query string\n",
    "    qs = paste0(colnames(pred),\"=\",pred[1,],collapse=\"&\")\n",
    "    \n",
    "    server = \"https://boiling-forest-8250.herokuapp.com/predict\"\n",
    "    \n",
    "    URL = paste0(server,\"?\",qs)\n",
    "    \n",
    "    js  = fromJSON(URL)\n",
    "    df = js$preds\n",
    "    df$college = as.factor(df$college)\n",
    " \n",
    "\n",
    "    \n",
    "    \n",
    "    #report\n",
    "    str1 <- paste(\"<p>Our algorithm predicts you have a<br>\")\n",
    "    str2 <- paste(\"percent chance of getting in to\")\n",
    "    str3 <- paste(\"<br><br>The 95% confidence interval on this prediction is\")\n",
    "    str4 <- paste(\"to\")\n",
    "    HTML(str1,100*df$prob[df$college==input$college],str2,input$college)\n",
    "    }\n",
    "       \n",
    "     })\n",
    "  \n",
    "  #echo importances\n",
    "  output$importance<- renderPlot({\n",
    "    \n",
    "    if (input$college!=\"\"){\n",
    "      \n",
    "      createAlert(session, \"alert\", \"Alert\", title = \"This prediction is not a guarantee of admission.\", \n",
    "                  \"We are simply interested in exploring how application factors affect college admissions. Our algorithm is 74% accurate.\", append = FALSE);\n",
    "      \n",
    "      drops <- c(\"finAidPct\",\"instatePct\")\n",
    "      newimportdf<-importdf[!(rownames(importdf) %in% drops),]\n",
    "      \n",
    "     \n",
    "      \n",
    "      \n",
    "    ggplot(data=newimportdf,aes(y=MeanDecreaseGini,x=reorder,fill=MeanDecreaseGini))+\n",
    "      geom_bar(stat=\"identity\")+coord_flip()+\n",
    "      ggtitle(paste('Importance of Application Components'))+\n",
    "      xlab(paste('Application Component'))+\n",
    "      ylab(paste('Effect Size'))+\n",
    "      scale_x_discrete(labels=better_labels)+\n",
    "      guides(fill=FALSE)+\n",
    "      theme(panel.grid.major = element_blank(), panel.grid.minor = element_blank(), \n",
    "            panel.background = element_blank(), axis.line = element_line(colour = \"black\"))\n",
    "    }\n",
    "    \n",
    "    \n",
    "  })\n",
    "  \n",
    "  output$importancehelper <-renderUI({\n",
    "    if (input$college!=\"\"){\n",
    "    HTML(\"This plot shows the relative importance of various parts of the application. In other words, when we train a model to predict your chance of admission, it weights the different aspects of your application according to these importances. Notice that SAT and ACT scores play the largest role in determining your chances of acceptance.\")\n",
    "    }\n",
    "      })\n",
    "  \n",
    "  output$scatterhelper <-renderUI({\n",
    "    if (input$college!=\"\"){\n",
    "      HTML(\"These graphs shows the relationship between an applicant's qualifications and their acceptance to\",input$college,\". Each dot represents a student in our data set; blue dots denote a student who was accepted.\")\n",
    "    }\n",
    "  })\n",
    "  \n",
    "  \n",
    "  #echo scatterplot\n",
    "  vis<- reactive({\n",
    "\n",
    "      # Lables for axes\n",
    "      xvar_name <- colnames(subdata)[colnames(subdata) == input$xvar]\n",
    "      yvar_name <- colnames(subdata)[colnames(subdata) == input$yvar]\n",
    "      \n",
    "      # Normally we could do something like props(x = ~GPA, y = ~SAT),\n",
    "      # but since the inputs are strings, we need to do a little more work.\n",
    "      xvar <- prop(\"x\", as.symbol(input$xvar))\n",
    "      yvar <- prop(\"y\", as.symbol(input$yvar))\n",
    "      \n",
    "      if (input$college!='')\n",
    "      {graphdata = dataunnormed[dataunnormed$name==input$college,]\n",
    "      graphdata$acceptStatus[graphdata$acceptStatus==-1]<-0}\n",
    "      else\n",
    "      {graphdata = subdata\n",
    "      graphdata$acceptStatus<-0}\n",
    "      \n",
    "      \n",
    "      graphdata %>%\n",
    "        ggvis(x = xvar, y = yvar) %>%\n",
    "        layer_points(size := 50, size.hover := 200,\n",
    "                     fillOpacity := 0.2, fillOpacity.hover := 0.5,\n",
    "                     stroke = ~acceptStatus) %>%\n",
    "        #layer_points(x = xvar, y= yvar, size := 50, fillOpacity=1,fill:=\"red\", data = testdata)%>%\n",
    "        add_axis(\"x\", title = xvar_name) %>%\n",
    "        add_axis(\"y\", title = yvar_name) %>%\n",
    "        add_legend(\"stroke\", title = \"Accepted\", values = c(\"Yes\", \"No\")) %>%\n",
    "        #scale_nominal(\"stroke\", domain = c(\"Yes\", \"No\"),\n",
    "                      #range = c(\"blue\", \"#aaa\")) %>%\n",
    "        set_options(width = 500, height = 500)\n",
    "      \n",
    "    \n",
    "  })\n",
    "  \n",
    "  vis %>% bind_shiny(\"plot1\")\n",
    "  \n",
    "\n",
    "  \n",
    "\n",
    "#echo link to CS109\n",
    "output$link <- renderUI({\n",
    "   return(h4(p(\"Want to see other cool machine learning tools and projects? Visit\",\n",
    "                strong(a(\"the Harvard CS109 homepage.\", href=\"http://cs109.github.io\")))))\n",
    "  \n",
    "\n",
    "})\n",
    "\n",
    "\n",
    "\n",
    "  ## PART THREE: alert\n",
    "  output$alert <- renderUI({\n",
    "   \n",
    "    \n",
    "        createAlert(session, \"alert\", \"Alert\", title = \"This is not a guarantee of admission!\", \n",
    "                    \"This website is experimental and exploratory. Feel free to toggle the settings to see how your chances might change with an extra AP class or two.\", append = FALSE);\n",
    "     \n",
    "  })\n",
    "\n",
    "  \n",
    "})\n",
    "\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"squarespace\"></a>\n",
    "## Part 3 - SquareSpace"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "SquareSpace hosts the static portion of the public facing web site. It also provides summary usage statistics."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "<a id=\"references\"></a>\n",
    "## References\n",
    "\n",
    "Getting Started with Python on Heroku https://devcenter.heroku.com/articles/getting-started-with-python-o#prerequisites\n",
    "\n",
    "Buildpack for Conda on Heroku https://github.com/kennethreitz/conda-buildpack\n",
    "\n",
    "Getting started with OpenShift and Python 2.7 (without Flask): https://developers.openshift.com/en/python-getting-started.html\n",
    "\n",
    "Getting started with OpenShift and Flask: https://developers.openshift.com/en/python-flask.html\n",
    "\n",
    "Blog post about OpenShift and Flask https://blog.openshift.com/day-3-flask-instant-python-web-development-with-python-and-openshift/\n",
    "\n",
    "Somewhat dated: https://blog.openshift.com/beginners-guide-to-writing-flask-apps-on-openshift/"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}