Simple Exploration Notebook - Zillow Prize 6889d9.ipynb

{"cells":[{"metadata":{"_cell_guid":"ae353a12-9edf-1946-c11c-88d539b113d0"},"cell_type":"markdown","source":"In this notebook, let us try and explore the data given for Zillow prize competition. Before we dive deep into the data, let us know a little more about the competition.\n\n**Zillow:**\n\nZillow is an online real estate database company founded in 2006 - Wikipedia\n\n**Zestimate:**\n\n“Zestimates” are estimated home values based on 7.5 million statistical and machine learning models that analyze hundreds of data points on each property. And, by continually improving the median margin of error (from 14% at the onset to 5% today),\n\n**Objective:**\n\nBuilding a model to improve the Zestimate residual error.\n\nThe competition is in two stages. This public competition will go on till Jan 2018 and has $50,000 in prize. Please make sure to read about the [Prize details][1] and [Competition overview][2] since it is quite different in this one.\n\nLet us first import the necessary modules.\n\n\n  [1]: https://www.kaggle.com/c/zillow-prize-1#prizes\n  [2]: https://www.kaggle.com/c/zillow-prize-1#Competition%20Overview"},{"metadata":{"_cell_guid":"7b0e7606-d591-ad0a-3340-7a98a6f0b519","trusted":false},"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pyplot as plt\nimport seaborn as sns\ncolor = sns.color_palette()\n\n%matplotlib inline\n\npd.options.mode.chained_assignment = None\npd.options.display.max_columns = 999","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"12443c4c-af3e-d890-9cd0-83e3e08fd6d2"},"cell_type":"markdown","source":"Let us list the files present in the input folder."},{"metadata":{"_cell_guid":"e9a846eb-7c8e-7c62-7299-4febbb5ebedf","trusted":false},"cell_type":"code","source":"from subprocess import check_output\nprint(check_output([\"ls\", \"../input\"]).decode(\"utf8\"))","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"9ed1b204-7eb1-ee23-073a-8085a51d0804"},"cell_type":"markdown","source":"First let us start with the train file exploration."},{"metadata":{"_cell_guid":"6ed52ad2-5d37-1c05-edca-5d200f497e7b","trusted":false},"cell_type":"code","source":"train_df = pd.read_csv(\"../input/train_2016.csv\", parse_dates=[\"transactiondate\"])\ntrain_df.shape","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"553701b1-1508-fb32-7212-4d6c4f6ab796","trusted":false},"cell_type":"code","source":"train_df.head()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"7a04bd96-470e-bfc3-8099-a0b6e774ea6a"},"cell_type":"markdown","source":"**Logerror:**\n\nTarget variable for this competition is \"logerror\" field. So let us do some analysis on this field first. "},{"metadata":{"_cell_guid":"ff5265e2-3630-a262-da96-4f289ffed677","trusted":false},"cell_type":"code","source":"plt.figure(figsize=(8,6))\nplt.scatter(range(train_df.shape[0]), np.sort(train_df.logerror.values))\nplt.xlabel('index', fontsize=12)\nplt.ylabel('logerror', fontsize=12)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"1cce93eb-0687-fcc0-983e-60a2ea59b87e"},"cell_type":"markdown","source":"This looks nice with some outliers at both the ends.! \n\nLet us remove the outliers and then do a histogram plot on the same."},{"metadata":{"_cell_guid":"67ad699f-2314-883d-6cf3-b15f2ebdc7d7","trusted":false},"cell_type":"code","source":"ulimit = np.percentile(train_df.logerror.values, 99)\nllimit = np.percentile(train_df.logerror.values, 1)\ntrain_df['logerror'].ix[train_df['logerror']>ulimit] = ulimit\ntrain_df['logerror'].ix[train_df['logerror']<llimit] = llimit\n\nplt.figure(figsize=(12,8))\nsns.distplot(train_df.logerror.values, bins=50, kde=False)\nplt.xlabel('logerror', fontsize=12)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"4ca4e0aa-8652-cb13-5ca8-88ef50794147"},"cell_type":"markdown","source":"Wow. nice normal distribution on the log error.\n\n**Transaction Date:**\n\nNow let us explore the date field. Let us first check the number of transactions in each month. "},{"metadata":{"_cell_guid":"2b641d53-1080-8141-47a9-065eb66ebd09","trusted":false},"cell_type":"code","source":"train_df['transaction_month'] = train_df['transactiondate'].dt.month\n\ncnt_srs = train_df['transaction_month'].value_counts()\nplt.figure(figsize=(12,6))\nsns.barplot(cnt_srs.index, cnt_srs.values, alpha=0.8, color=color[3])\nplt.xticks(rotation='vertical')\nplt.xlabel('Month of transaction', fontsize=12)\nplt.ylabel('Number of Occurrences', fontsize=12)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"7d525f89-1091-d0a7-1dc2-ade5c5027259"},"cell_type":"markdown","source":"As we could see from the data page as well\n*The train data has all the transactions before October 15, 2016, plus some of the transactions after October 15, 2016.*\n\nSo we have shorter bars in the last three months. \n\n**Parcel Id:**"},{"metadata":{"_cell_guid":"e3055a80-3151-0a0e-c388-9c77105cea8c","trusted":false},"cell_type":"code","source":"(train_df['parcelid'].value_counts().reset_index())['parcelid'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"b250fd92-b9ac-da88-5faa-69c22ef142b4"},"cell_type":"markdown","source":"So most of the parcel ids are appearing only once in the dataset.\n\n**Properties 2016:**\n\nNow let us explore the properties_2016 file. "},{"metadata":{"_cell_guid":"e356e5d9-dc5a-a1c7-15d4-47a4b3af1075","trusted":false},"cell_type":"code","source":"prop_df = pd.read_csv(\"../input/properties_2016.csv\")\nprop_df.shape","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"0a777643-d6d0-e2cd-e2a0-654a97034001","trusted":false},"cell_type":"code","source":"prop_df.head()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"5a4e7b91-5ee2-f972-af70-08e660e93162"},"cell_type":"markdown","source":"There are so many NaN values in the dataset. So let us first do some exploration on that one. "},{"metadata":{"_cell_guid":"3f13cac6-5d01-fc2d-73be-98472e5a988a","trusted":false},"cell_type":"code","source":"missing_df = prop_df.isnull().sum(axis=0).reset_index()\nmissing_df.columns = ['column_name', 'missing_count']\nmissing_df = missing_df.ix[missing_df['missing_count']>0]\nmissing_df = missing_df.sort_values(by='missing_count')\n\nind = np.arange(missing_df.shape[0])\nwidth = 0.9\nfig, ax = plt.subplots(figsize=(12,18))\nrects = ax.barh(ind, missing_df.missing_count.values, color='blue')\nax.set_yticks(ind)\nax.set_yticklabels(missing_df.column_name.values, rotation='horizontal')\nax.set_xlabel(\"Count of missing values\")\nax.set_title(\"Number of missing values in each column\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"c9eda5a2-1c29-f7c0-36bc-07aa6b8d4499"},"cell_type":"markdown","source":"Let us explore the latitude and longitude variable to begin with."},{"metadata":{"_cell_guid":"e33a5c3d-1c0e-ebf9-03fd-ff3a941954d7","trusted":false},"cell_type":"code","source":"plt.figure(figsize=(12,12))\nsns.jointplot(x=prop_df.latitude.values, y=prop_df.longitude.values, size=10)\nplt.ylabel('Longitude', fontsize=12)\nplt.xlabel('Latitude', fontsize=12)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"412ae7df-df18-4f37-a324-a30bc231cbd3"},"cell_type":"markdown","source":"From the data page, *we are provided with a full list of real estate properties in three counties (Los Angeles, Orange and Ventura, California) data in 2016.*\n\nWe have about 90,811 rows in train but we have about 2,985,217 rows in properties file. So let us merge the two files and then carry out our analysis. "},{"metadata":{"_cell_guid":"aacb09c5-1009-a689-a49c-b89a7a7e3826","trusted":false},"cell_type":"code","source":"train_df = pd.merge(train_df, prop_df, on='parcelid', how='left')\ntrain_df.head()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"fcbb8b2a-9040-7fd8-d08d-f887821f85d5"},"cell_type":"markdown","source":"Now let us check the dtypes of different types of variable."},{"metadata":{"_cell_guid":"b4131373-35f7-7819-1cf0-d8fdf55589c8","trusted":false},"cell_type":"code","source":"pd.options.display.max_rows = 65\n\ndtype_df = train_df.dtypes.reset_index()\ndtype_df.columns = [\"Count\", \"Column Type\"]\ndtype_df","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"ec4a5fd5-95e6-e564-3b8e-28065cb5e3b4"},"cell_type":"markdown","source":"Almost all are float variables with few object (categorical) variables. Let us get the count. "},{"metadata":{"_cell_guid":"2d546cc2-c74d-afdc-a678-29f59f78f6da","trusted":false},"cell_type":"code","source":"dtype_df.groupby(\"Column Type\").aggregate('count').reset_index()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"95ea35b7-f1cd-d8a5-d719-5dfd8d012e75"},"cell_type":"markdown","source":"Now let us check the number of Nulls in this new merged dataset."},{"metadata":{"_cell_guid":"e56a107d-91fd-aae0-9cc4-f88604ca81df","trusted":false},"cell_type":"code","source":"missing_df = train_df.isnull().sum(axis=0).reset_index()\nmissing_df.columns = ['column_name', 'missing_count']\nmissing_df['missing_ratio'] = missing_df['missing_count'] / train_df.shape[0]\nmissing_df.ix[missing_df['missing_ratio']>0.999]","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"c365beb4-7d74-b36f-187d-7c4c3c288dfa"},"cell_type":"markdown","source":"Four columns have missing values 99.9% of the times.!"},{"metadata":{"_cell_guid":"2abd4104-622c-9ada-b397-b756664f69d3"},"cell_type":"markdown","source":"**Univariate Analysis:**\n\nSince there are so many variables, let us first take the 'float' variables alone and then get the correlation with the target variable to see how they are related."},{"metadata":{"_cell_guid":"a6592dca-8f7a-cd6b-23df-d87a98ebf934","trusted":false},"cell_type":"code","source":"# Let us just impute the missing values with mean values to compute correlation coefficients #\nmean_values = train_df.mean(axis=0)\ntrain_df_new = train_df.fillna(mean_values, inplace=True)\n\n# Now let us look at the correlation coefficient of each of these variables #\nx_cols = [col for col in train_df_new.columns if col not in ['logerror'] if train_df_new[col].dtype=='float64']\n\nlabels = []\nvalues = []\nfor col in x_cols:\n    labels.append(col)\n    values.append(np.corrcoef(train_df_new[col].values, train_df_new.logerror.values)[0,1])\ncorr_df = pd.DataFrame({'col_labels':labels, 'corr_values':values})\ncorr_df = corr_df.sort_values(by='corr_values')\n    \nind = np.arange(len(labels))\nwidth = 0.9\nfig, ax = plt.subplots(figsize=(12,40))\nrects = ax.barh(ind, np.array(corr_df.corr_values.values), color='y')\nax.set_yticks(ind)\nax.set_yticklabels(corr_df.col_labels.values, rotation='horizontal')\nax.set_xlabel(\"Correlation coefficient\")\nax.set_title(\"Correlation coefficient of the variables\")\n#autolabel(rects)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"92410577-02ba-d92d-b6b3-3673819ede1f"},"cell_type":"markdown","source":"The correlation of the target variable with the given set of variables are low overall. \n\nThere are few variables at the top of this graph without any correlation values. I guess they have only one unique value and hence no correlation value. Let us confirm the same."},{"metadata":{"_cell_guid":"2331b242-994c-8865-4fc8-13a62f518788","trusted":false},"cell_type":"code","source":"corr_zero_cols = ['assessmentyear', 'storytypeid', 'pooltypeid2', 'pooltypeid7', 'pooltypeid10', 'poolcnt', 'decktypeid', 'buildingclasstypeid']\nfor col in corr_zero_cols:\n    print(col, len(train_df_new[col].unique()))","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"927f3dfd-3461-a058-e70d-b23719b45b57"},"cell_type":"markdown","source":"Let us take the variables with high correlation values and then do some analysis on them. "},{"metadata":{"_cell_guid":"7971e8c3-69c3-7797-9ab4-04d60f2cc889","trusted":false},"cell_type":"code","source":"corr_df_sel = corr_df.ix[(corr_df['corr_values']>0.02) | (corr_df['corr_values'] < -0.01)]\ncorr_df_sel","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"78e253cf-ed6a-4e64-834d-f8c2c2ac1788","trusted":false},"cell_type":"code","source":"cols_to_use = corr_df_sel.col_labels.tolist()\n\ntemp_df = train_df[cols_to_use]\ncorrmat = temp_df.corr(method='spearman')\nf, ax = plt.subplots(figsize=(8, 8))\n\n# Draw the heatmap using seaborn\nsns.heatmap(corrmat, vmax=1., square=True)\nplt.title(\"Important variables correlation map\", fontsize=15)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"87caaa5f-ed65-6ec1-75e8-af450304f966"},"cell_type":"markdown","source":"The important variables themselves are very highly correlated.! Let us now look at each of them.\n\n**Finished SquareFeet 12:**\n\nLet us seee how the finished square feet 12 varies with the log error."},{"metadata":{"_cell_guid":"9a051a83-6f9b-e8ba-35b7-a84911d73b6b","trusted":false},"cell_type":"code","source":"col = \"finishedsquarefeet12\"\nulimit = np.percentile(train_df[col].values, 99.5)\nllimit = np.percentile(train_df[col].values, 0.5)\ntrain_df[col].ix[train_df[col]>ulimit] = ulimit\ntrain_df[col].ix[train_df[col]<llimit] = llimit\n\nplt.figure(figsize=(12,12))\nsns.jointplot(x=train_df.finishedsquarefeet12.values, y=train_df.logerror.values, size=10, color=color[4])\nplt.ylabel('Log Error', fontsize=12)\nplt.xlabel('Finished Square Feet 12', fontsize=12)\nplt.title(\"Finished square feet 12 Vs Log error\", fontsize=15)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"a47ef671-9fc4-91eb-266c-e5c41a1af987"},"cell_type":"markdown","source":"Seems the range of logerror narrows down with increase in finished square feet 12 variable. Probably larger houses are easy to predict?\n\n**Calculated finished square feet:**"},{"metadata":{"_cell_guid":"4f782bf2-5753-9017-eab8-e0c8cc0e1236","trusted":false},"cell_type":"code","source":"col = \"calculatedfinishedsquarefeet\"\nulimit = np.percentile(train_df[col].values, 99.5)\nllimit = np.percentile(train_df[col].values, 0.5)\ntrain_df[col].ix[train_df[col]>ulimit] = ulimit\ntrain_df[col].ix[train_df[col]<llimit] = llimit\n\nplt.figure(figsize=(12,12))\nsns.jointplot(x=train_df.calculatedfinishedsquarefeet.values, y=train_df.logerror.values, size=10, color=color[5])\nplt.ylabel('Log Error', fontsize=12)\nplt.xlabel('Calculated finished square feet', fontsize=12)\nplt.title(\"Calculated finished square feet Vs Log error\", fontsize=15)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"936de827-6302-efb1-993e-be2c113e28bd"},"cell_type":"markdown","source":"Here as well the distribution is very similar to the previous one. No wonder the correlation between the two variables are also high.\n\n**Bathroom Count:**"},{"metadata":{"_cell_guid":"6fd53daa-e61a-5270-d41d-d8fa7e7e603a","trusted":false},"cell_type":"code","source":"plt.figure(figsize=(12,8))\nsns.countplot(x=\"bathroomcnt\", data=train_df)\nplt.ylabel('Count', fontsize=12)\nplt.xlabel('Bathroom', fontsize=12)\nplt.xticks(rotation='vertical')\nplt.title(\"Frequency of Bathroom count\", fontsize=15)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"90e9db54-5167-5f2f-60da-139802ade846"},"cell_type":"markdown","source":"There is an interesting 2.279 value in the bathroom count.\n\nEdit: As MihwaHan pointed in the comments, this is the mean value :)\n\nNow let us check how the log error changes based on this."},{"metadata":{"_cell_guid":"5709506f-3dad-fe82-2207-f83fa84662e8","trusted":false},"cell_type":"code","source":"plt.figure(figsize=(12,8))\nsns.boxplot(x=\"bathroomcnt\", y=\"logerror\", data=train_df)\nplt.ylabel('Log error', fontsize=12)\nplt.xlabel('Bathroom Count', fontsize=12)\nplt.xticks(rotation='vertical')\nplt.title(\"How log error changes with bathroom count?\", fontsize=15)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"f3348b3a-0276-6a79-afb2-5183ae4b8f66"},"cell_type":"markdown","source":"**Bedroom count:**"},{"metadata":{"_cell_guid":"89d6cc37-8a90-34fe-d427-feaeb3c3a6c2","trusted":false},"cell_type":"code","source":"plt.figure(figsize=(12,8))\nsns.countplot(x=\"bedroomcnt\", data=train_df)\nplt.ylabel('Frequency', fontsize=12)\nplt.xlabel('Bedroom Count', fontsize=12)\nplt.xticks(rotation='vertical')\nplt.title(\"Frequency of Bedroom count\", fontsize=15)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"7e63558e-1036-764d-8c11-47c4f5e8bfc0"},"cell_type":"markdown","source":"3.03 is the mean value with which we replaced the Null values."},{"metadata":{"_cell_guid":"957f4ad5-4043-fcfe-3c30-d3dbac494095","trusted":false},"cell_type":"code","source":"train_df['bedroomcnt'].ix[train_df['bedroomcnt']>7] = 7\nplt.figure(figsize=(12,8))\nsns.violinplot(x='bedroomcnt', y='logerror', data=train_df)\nplt.xlabel('Bedroom count', fontsize=12)\nplt.ylabel('Log Error', fontsize=12)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"acac05b8-70d8-9e65-1b83-9adb2406d4a3","trusted":false},"cell_type":"code","source":"col = \"taxamount\"\nulimit = np.percentile(train_df[col].values, 99.5)\nllimit = np.percentile(train_df[col].values, 0.5)\ntrain_df[col].ix[train_df[col]>ulimit] = ulimit\ntrain_df[col].ix[train_df[col]<llimit] = llimit\n\nplt.figure(figsize=(12,12))\nsns.jointplot(x=train_df['taxamount'].values, y=train_df['logerror'].values, size=10, color='g')\nplt.ylabel('Log Error', fontsize=12)\nplt.xlabel('Tax Amount', fontsize=12)\nplt.title(\"Tax Amount Vs Log error\", fontsize=15)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"e0a42282-764a-1776-6dde-6bf238e6d87d"},"cell_type":"markdown","source":"**YearBuilt:**\n\nLet us explore how the error varies with the yearbuilt variable."},{"metadata":{"_cell_guid":"dcd10464-6b91-f0f2-5c4d-ad654fd60b32","trusted":false},"cell_type":"code","source":"from ggplot import *\nggplot(aes(x='yearbuilt', y='logerror'), data=train_df) + \\\n    geom_point(color='steelblue', size=1) + \\\n    stat_smooth()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"7461d6f6-4dca-25d8-ee58-51a9068926d8"},"cell_type":"markdown","source":"There is a minor incremental trend seen with respect to built year.\n\nNow let us see how the logerror varies with respect to latitude and longitude."},{"metadata":{"_cell_guid":"fe919c48-a6e3-e6b2-601f-09ad3e568269","trusted":false},"cell_type":"code","source":"ggplot(aes(x='latitude', y='longitude', color='logerror'), data=train_df) + \\\n    geom_point() + \\\n    scale_color_gradient(low = 'red', high = 'blue')","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"be81b990-494e-5e75-df67-18548f650b29"},"cell_type":"markdown","source":"There are no visible pockets as such with respect to latitude or longitude atleast with the naked eye.\n\nLet us take the variables with highest positive correlation and highest negative correlation to see if we can see some visible patterns."},{"metadata":{"_cell_guid":"ed814938-7acf-ec9c-5070-22ddad06dbfc","trusted":false},"cell_type":"code","source":"ggplot(aes(x='finishedsquarefeet12', y='taxamount', color='logerror'), data=train_df) + \\\n    geom_point(alpha=0.7) + \\\n    scale_color_gradient(low = 'pink', high = 'blue')","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"aecf530d-c28b-ea61-c11a-63005f3e1e22"},"cell_type":"markdown","source":"There are no visible patterns here as well. So this is going to be a hard competition to predict I guess.\n\nJust for fun, we will let the machine form some arbitrary pattern for us :D"},{"metadata":{"_cell_guid":"27b18ae9-f711-7b8b-c42f-9133f56340fa","trusted":false},"cell_type":"code","source":"ggplot(aes(x='finishedsquarefeet12', y='taxamount', color='logerror'), data=train_df) + \\\n    geom_now_its_art()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"8b608db2-b0fe-0e7b-ffee-6f5b9509afbb"},"cell_type":"markdown","source":"Hurray.! Finally we got some nice pattern in the data :P\n\nWe had an understanding of important variables from the univariate analysis. But this is on a stand alone basis and also we have linearity assumption. Now let us build a non-linear model to get the important variables by building Extra Trees model."},{"metadata":{"_cell_guid":"91013c8a-9980-62c0-bc4e-e9a2d3e251d7","trusted":false},"cell_type":"code","source":"train_y = train_df['logerror'].values\ncat_cols = [\"hashottuborspa\", \"propertycountylandusecode\", \"propertyzoningdesc\", \"fireplaceflag\", \"taxdelinquencyflag\"]\ntrain_df = train_df.drop(['parcelid', 'logerror', 'transactiondate', 'transaction_month']+cat_cols, axis=1)\nfeat_names = train_df.columns.values\n\nfrom sklearn import ensemble\nmodel = ensemble.ExtraTreesRegressor(n_estimators=25, max_depth=30, max_features=0.3, n_jobs=-1, random_state=0)\nmodel.fit(train_df, train_y)\n\n## plot the importances ##\nimportances = model.feature_importances_\nstd = np.std([tree.feature_importances_ for tree in model.estimators_], axis=0)\nindices = np.argsort(importances)[::-1][:20]\n\nplt.figure(figsize=(12,12))\nplt.title(\"Feature importances\")\nplt.bar(range(len(indices)), importances[indices], color=\"r\", yerr=std[indices], align=\"center\")\nplt.xticks(range(len(indices)), feat_names[indices], rotation='vertical')\nplt.xlim([-1, len(indices)])\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"3bc28a7d-cb65-58e1-c7fc-0df34998d7a2"},"cell_type":"markdown","source":"Seems \"tax amount\" is the most importanct variable followed by \"structure tax value dollar count\" and \"land tax value dollor count\"\n\n"},{"metadata":{"_cell_guid":"e59aeef7-7ec8-0b26-0897-40371a4cda24","trusted":false},"cell_type":"code","source":"import xgboost as xgb\nxgb_params = {\n    'eta': 0.05,\n    'max_depth': 8,\n    'subsample': 0.7,\n    'colsample_bytree': 0.7,\n    'objective': 'reg:linear',\n    'silent': 1,\n    'seed' : 0\n}\ndtrain = xgb.DMatrix(train_df, train_y, feature_names=train_df.columns.values)\nmodel = xgb.train(dict(xgb_params, silent=0), dtrain, num_boost_round=50)\n\n# plot the important features #\nfig, ax = plt.subplots(figsize=(12,18))\nxgb.plot_importance(model, max_num_features=50, height=0.8, ax=ax)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"cf80ce26-eef4-6b17-c152-973510ab547d"},"cell_type":"markdown","source":"Using xgboost, the important variables are 'structured tax value dollar count' followed by 'latitude' and 'calculated finished square feet' "},{"metadata":{"_cell_guid":"5026922d-4f83-c0fd-bb20-26d5a8e9224a"},"cell_type":"markdown","source":"**More to come. Stay tuned.!**\n\n**Please upvote if you find it useful :)**"}],"metadata":{"_change_revision":0,"_is_fork":false,"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.6.0"}},"nbformat":4,"nbformat_minor":1}