clustering.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Clustering (also known as cluster analysis) is a quintessential method of unsupervised machine learning. It's primarily used for dividing data up into groups (or clusters) based on shared criteria. By examining these clusters, different \"families\" can be deduced, families which provide a useful category to assign future data points to. For our purposes, clustering is used to divide up VBA macros based on their content, clusters to which we're able to assign varying levels of \"maliciousnous\" or \"benignness.\" What follows is a demonstration of this process, with clusters formed based on the words contained in a macro.\n",
    "Read the full blog at [InQuest Blog: Clustering for Classification](https://inquest.net/blog/2020/12/16/Clustering-for-Classification)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. First, we must import everything we need. This includes numpy and pandas for opening and reading our dataframes, and our clustering algorithm. For our clustering algorithm, we use sklearn's kmeans classifier. We also load in our data into a panda's dataframe. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>abs</th>\n",
       "      <th>accelerator</th>\n",
       "      <th>activate</th>\n",
       "      <th>activecell</th>\n",
       "      <th>activecodepane</th>\n",
       "      <th>activecontrol</th>\n",
       "      <th>activedocument</th>\n",
       "      <th>activesheet</th>\n",
       "      <th>activevbproject</th>\n",
       "      <th>activewindow</th>\n",
       "      <th>...</th>\n",
       "      <th>Join</th>\n",
       "      <th>LBound</th>\n",
       "      <th>Split</th>\n",
       "      <th>UBound</th>\n",
       "      <th>CurDir</th>\n",
       "      <th>Dir</th>\n",
       "      <th>FileAttr</th>\n",
       "      <th>FileDateTime</th>\n",
       "      <th>FileLen</th>\n",
       "      <th>GetAttr</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>4ee2939230a5962bc7937f0e54c27900595c4040ba723f6f31d84cc9e60ac3a7.macro</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.010344</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.106488</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7b409025db2a30a58b42adc20bc81daf7dd6a6f8b7dfd089e90fb9fb232a5c05.macro</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.002899</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2cba5a34d84cba315019a94f15a70ba3f2b013955cd240ccca106aa9a569f827.macro</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>e406765eedf8d315750823133c94acb9d8af9a74afb25af8f983a5d4d48af5b7.macro</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.281521</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3e1b48c1b05736c3125723decf7cfe0e4e4efa51de63ad11eda1fa35e564b72e.macro</th>\n",
       "      <td>0.008199</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.008457</td>\n",
       "      <td>0.016246</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.021371</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>f58a9e939d186b69453367e26d1123dc1a03719c8f3c0f69a5157c8a8faca814.macro</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4df4ed09ddcf6e2aa6d682bae21f3b2c243551310bdd565c2d2e0e89f05c4bd6.macro</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.020384</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.028581</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>a5c6ab5f55851e5a4a665888b4804457a9c27b82bfdadd98185dc733b6ab6ed4.macro</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>985577430d9c17e113d00a3923df0e4e6bcf452452fdd2e3196f89f9b5cffb34.macro</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>540ef136b0435594562ea3596c65104a42977047590f2aebfdde8dae09d42187.macro</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.017944</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10002 rows × 897 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                         abs  accelerator  \\\n",
       "4ee2939230a5962bc7937f0e54c27900595c4040ba723f6...  0.000000          0.0   \n",
       "7b409025db2a30a58b42adc20bc81daf7dd6a6f8b7dfd08...  0.000000          0.0   \n",
       "2cba5a34d84cba315019a94f15a70ba3f2b013955cd240c...  0.000000          0.0   \n",
       "e406765eedf8d315750823133c94acb9d8af9a74afb25af...  0.000000          0.0   \n",
       "3e1b48c1b05736c3125723decf7cfe0e4e4efa51de63ad1...  0.008199          0.0   \n",
       "...                                                      ...          ...   \n",
       "f58a9e939d186b69453367e26d1123dc1a03719c8f3c0f6...  0.000000          0.0   \n",
       "4df4ed09ddcf6e2aa6d682bae21f3b2c243551310bdd565...  0.000000          0.0   \n",
       "a5c6ab5f55851e5a4a665888b4804457a9c27b82bfdadd9...  0.000000          0.0   \n",
       "985577430d9c17e113d00a3923df0e4e6bcf452452fdd2e...  0.000000          0.0   \n",
       "540ef136b0435594562ea3596c65104a42977047590f2ae...  0.000000          0.0   \n",
       "\n",
       "                                                    activate  activecell  \\\n",
       "4ee2939230a5962bc7937f0e54c27900595c4040ba723f6...  0.010344    0.000000   \n",
       "7b409025db2a30a58b42adc20bc81daf7dd6a6f8b7dfd08...  0.000000    0.000000   \n",
       "2cba5a34d84cba315019a94f15a70ba3f2b013955cd240c...  0.000000    0.000000   \n",
       "e406765eedf8d315750823133c94acb9d8af9a74afb25af...  0.000000    0.000000   \n",
       "3e1b48c1b05736c3125723decf7cfe0e4e4efa51de63ad1...  0.008457    0.016246   \n",
       "...                                                      ...         ...   \n",
       "f58a9e939d186b69453367e26d1123dc1a03719c8f3c0f6...  0.000000    0.000000   \n",
       "4df4ed09ddcf6e2aa6d682bae21f3b2c243551310bdd565...  0.020384    0.000000   \n",
       "a5c6ab5f55851e5a4a665888b4804457a9c27b82bfdadd9...  0.000000    0.000000   \n",
       "985577430d9c17e113d00a3923df0e4e6bcf452452fdd2e...  0.000000    0.000000   \n",
       "540ef136b0435594562ea3596c65104a42977047590f2ae...  0.017944    0.000000   \n",
       "\n",
       "                                                    activecodepane  \\\n",
       "4ee2939230a5962bc7937f0e54c27900595c4040ba723f6...             0.0   \n",
       "7b409025db2a30a58b42adc20bc81daf7dd6a6f8b7dfd08...             0.0   \n",
       "2cba5a34d84cba315019a94f15a70ba3f2b013955cd240c...             0.0   \n",
       "e406765eedf8d315750823133c94acb9d8af9a74afb25af...             0.0   \n",
       "3e1b48c1b05736c3125723decf7cfe0e4e4efa51de63ad1...             0.0   \n",
       "...                                                            ...   \n",
       "f58a9e939d186b69453367e26d1123dc1a03719c8f3c0f6...             0.0   \n",
       "4df4ed09ddcf6e2aa6d682bae21f3b2c243551310bdd565...             0.0   \n",
       "a5c6ab5f55851e5a4a665888b4804457a9c27b82bfdadd9...             0.0   \n",
       "985577430d9c17e113d00a3923df0e4e6bcf452452fdd2e...             0.0   \n",
       "540ef136b0435594562ea3596c65104a42977047590f2ae...             0.0   \n",
       "\n",
       "                                                    activecontrol  \\\n",
       "4ee2939230a5962bc7937f0e54c27900595c4040ba723f6...            0.0   \n",
       "7b409025db2a30a58b42adc20bc81daf7dd6a6f8b7dfd08...            0.0   \n",
       "2cba5a34d84cba315019a94f15a70ba3f2b013955cd240c...            0.0   \n",
       "e406765eedf8d315750823133c94acb9d8af9a74afb25af...            0.0   \n",
       "3e1b48c1b05736c3125723decf7cfe0e4e4efa51de63ad1...            0.0   \n",
       "...                                                           ...   \n",
       "f58a9e939d186b69453367e26d1123dc1a03719c8f3c0f6...            0.0   \n",
       "4df4ed09ddcf6e2aa6d682bae21f3b2c243551310bdd565...            0.0   \n",
       "a5c6ab5f55851e5a4a665888b4804457a9c27b82bfdadd9...            0.0   \n",
       "985577430d9c17e113d00a3923df0e4e6bcf452452fdd2e...            0.0   \n",
       "540ef136b0435594562ea3596c65104a42977047590f2ae...            0.0   \n",
       "\n",
       "                                                    activedocument  \\\n",
       "4ee2939230a5962bc7937f0e54c27900595c4040ba723f6...        0.000000   \n",
       "7b409025db2a30a58b42adc20bc81daf7dd6a6f8b7dfd08...        0.002899   \n",
       "2cba5a34d84cba315019a94f15a70ba3f2b013955cd240c...        0.000000   \n",
       "e406765eedf8d315750823133c94acb9d8af9a74afb25af...        0.281521   \n",
       "3e1b48c1b05736c3125723decf7cfe0e4e4efa51de63ad1...        0.000000   \n",
       "...                                                            ...   \n",
       "f58a9e939d186b69453367e26d1123dc1a03719c8f3c0f6...        0.000000   \n",
       "4df4ed09ddcf6e2aa6d682bae21f3b2c243551310bdd565...        0.028581   \n",
       "a5c6ab5f55851e5a4a665888b4804457a9c27b82bfdadd9...        0.000000   \n",
       "985577430d9c17e113d00a3923df0e4e6bcf452452fdd2e...        0.000000   \n",
       "540ef136b0435594562ea3596c65104a42977047590f2ae...        0.000000   \n",
       "\n",
       "                                                    activesheet  \\\n",
       "4ee2939230a5962bc7937f0e54c27900595c4040ba723f6...     0.106488   \n",
       "7b409025db2a30a58b42adc20bc81daf7dd6a6f8b7dfd08...     0.000000   \n",
       "2cba5a34d84cba315019a94f15a70ba3f2b013955cd240c...     0.000000   \n",
       "e406765eedf8d315750823133c94acb9d8af9a74afb25af...     0.000000   \n",
       "3e1b48c1b05736c3125723decf7cfe0e4e4efa51de63ad1...     0.021371   \n",
       "...                                                         ...   \n",
       "f58a9e939d186b69453367e26d1123dc1a03719c8f3c0f6...     0.000000   \n",
       "4df4ed09ddcf6e2aa6d682bae21f3b2c243551310bdd565...     0.000000   \n",
       "a5c6ab5f55851e5a4a665888b4804457a9c27b82bfdadd9...     0.000000   \n",
       "985577430d9c17e113d00a3923df0e4e6bcf452452fdd2e...     0.000000   \n",
       "540ef136b0435594562ea3596c65104a42977047590f2ae...     0.000000   \n",
       "\n",
       "                                                    activevbproject  \\\n",
       "4ee2939230a5962bc7937f0e54c27900595c4040ba723f6...              0.0   \n",
       "7b409025db2a30a58b42adc20bc81daf7dd6a6f8b7dfd08...              0.0   \n",
       "2cba5a34d84cba315019a94f15a70ba3f2b013955cd240c...              0.0   \n",
       "e406765eedf8d315750823133c94acb9d8af9a74afb25af...              0.0   \n",
       "3e1b48c1b05736c3125723decf7cfe0e4e4efa51de63ad1...              0.0   \n",
       "...                                                             ...   \n",
       "f58a9e939d186b69453367e26d1123dc1a03719c8f3c0f6...              0.0   \n",
       "4df4ed09ddcf6e2aa6d682bae21f3b2c243551310bdd565...              0.0   \n",
       "a5c6ab5f55851e5a4a665888b4804457a9c27b82bfdadd9...              0.0   \n",
       "985577430d9c17e113d00a3923df0e4e6bcf452452fdd2e...              0.0   \n",
       "540ef136b0435594562ea3596c65104a42977047590f2ae...              0.0   \n",
       "\n",
       "                                                    activewindow  ...  Join  \\\n",
       "4ee2939230a5962bc7937f0e54c27900595c4040ba723f6...           0.0  ...   0.0   \n",
       "7b409025db2a30a58b42adc20bc81daf7dd6a6f8b7dfd08...           0.0  ...   0.0   \n",
       "2cba5a34d84cba315019a94f15a70ba3f2b013955cd240c...           0.0  ...   0.0   \n",
       "e406765eedf8d315750823133c94acb9d8af9a74afb25af...           0.0  ...   0.0   \n",
       "3e1b48c1b05736c3125723decf7cfe0e4e4efa51de63ad1...           0.0  ...   0.0   \n",
       "...                                                          ...  ...   ...   \n",
       "f58a9e939d186b69453367e26d1123dc1a03719c8f3c0f6...           0.0  ...   0.0   \n",
       "4df4ed09ddcf6e2aa6d682bae21f3b2c243551310bdd565...           0.0  ...   0.0   \n",
       "a5c6ab5f55851e5a4a665888b4804457a9c27b82bfdadd9...           0.0  ...   0.0   \n",
       "985577430d9c17e113d00a3923df0e4e6bcf452452fdd2e...           0.0  ...   0.0   \n",
       "540ef136b0435594562ea3596c65104a42977047590f2ae...           0.0  ...   0.0   \n",
       "\n",
       "                                                    LBound  Split  UBound  \\\n",
       "4ee2939230a5962bc7937f0e54c27900595c4040ba723f6...     0.0    0.0     0.0   \n",
       "7b409025db2a30a58b42adc20bc81daf7dd6a6f8b7dfd08...     0.0    0.0     0.0   \n",
       "2cba5a34d84cba315019a94f15a70ba3f2b013955cd240c...     0.0    0.0     0.0   \n",
       "e406765eedf8d315750823133c94acb9d8af9a74afb25af...     0.0    0.0     0.0   \n",
       "3e1b48c1b05736c3125723decf7cfe0e4e4efa51de63ad1...     0.0    0.0     0.0   \n",
       "...                                                    ...    ...     ...   \n",
       "f58a9e939d186b69453367e26d1123dc1a03719c8f3c0f6...     0.0    0.0     0.0   \n",
       "4df4ed09ddcf6e2aa6d682bae21f3b2c243551310bdd565...     0.0    0.0     0.0   \n",
       "a5c6ab5f55851e5a4a665888b4804457a9c27b82bfdadd9...     0.0    0.0     0.0   \n",
       "985577430d9c17e113d00a3923df0e4e6bcf452452fdd2e...     0.0    0.0     0.0   \n",
       "540ef136b0435594562ea3596c65104a42977047590f2ae...     0.0    0.0     0.0   \n",
       "\n",
       "                                                    CurDir  Dir  FileAttr  \\\n",
       "4ee2939230a5962bc7937f0e54c27900595c4040ba723f6...     0.0  0.0       0.0   \n",
       "7b409025db2a30a58b42adc20bc81daf7dd6a6f8b7dfd08...     0.0  0.0       0.0   \n",
       "2cba5a34d84cba315019a94f15a70ba3f2b013955cd240c...     0.0  0.0       0.0   \n",
       "e406765eedf8d315750823133c94acb9d8af9a74afb25af...     0.0  0.0       0.0   \n",
       "3e1b48c1b05736c3125723decf7cfe0e4e4efa51de63ad1...     0.0  0.0       0.0   \n",
       "...                                                    ...  ...       ...   \n",
       "f58a9e939d186b69453367e26d1123dc1a03719c8f3c0f6...     0.0  0.0       0.0   \n",
       "4df4ed09ddcf6e2aa6d682bae21f3b2c243551310bdd565...     0.0  0.0       0.0   \n",
       "a5c6ab5f55851e5a4a665888b4804457a9c27b82bfdadd9...     0.0  0.0       0.0   \n",
       "985577430d9c17e113d00a3923df0e4e6bcf452452fdd2e...     0.0  0.0       0.0   \n",
       "540ef136b0435594562ea3596c65104a42977047590f2ae...     0.0  0.0       0.0   \n",
       "\n",
       "                                                    FileDateTime  FileLen  \\\n",
       "4ee2939230a5962bc7937f0e54c27900595c4040ba723f6...           0.0      0.0   \n",
       "7b409025db2a30a58b42adc20bc81daf7dd6a6f8b7dfd08...           0.0      0.0   \n",
       "2cba5a34d84cba315019a94f15a70ba3f2b013955cd240c...           0.0      0.0   \n",
       "e406765eedf8d315750823133c94acb9d8af9a74afb25af...           0.0      0.0   \n",
       "3e1b48c1b05736c3125723decf7cfe0e4e4efa51de63ad1...           0.0      0.0   \n",
       "...                                                          ...      ...   \n",
       "f58a9e939d186b69453367e26d1123dc1a03719c8f3c0f6...           0.0      0.0   \n",
       "4df4ed09ddcf6e2aa6d682bae21f3b2c243551310bdd565...           0.0      0.0   \n",
       "a5c6ab5f55851e5a4a665888b4804457a9c27b82bfdadd9...           0.0      0.0   \n",
       "985577430d9c17e113d00a3923df0e4e6bcf452452fdd2e...           0.0      0.0   \n",
       "540ef136b0435594562ea3596c65104a42977047590f2ae...           0.0      0.0   \n",
       "\n",
       "                                                    GetAttr  \n",
       "4ee2939230a5962bc7937f0e54c27900595c4040ba723f6...      0.0  \n",
       "7b409025db2a30a58b42adc20bc81daf7dd6a6f8b7dfd08...      0.0  \n",
       "2cba5a34d84cba315019a94f15a70ba3f2b013955cd240c...      0.0  \n",
       "e406765eedf8d315750823133c94acb9d8af9a74afb25af...      0.0  \n",
       "3e1b48c1b05736c3125723decf7cfe0e4e4efa51de63ad1...      0.0  \n",
       "...                                                     ...  \n",
       "f58a9e939d186b69453367e26d1123dc1a03719c8f3c0f6...      0.0  \n",
       "4df4ed09ddcf6e2aa6d682bae21f3b2c243551310bdd565...      0.0  \n",
       "a5c6ab5f55851e5a4a665888b4804457a9c27b82bfdadd9...      0.0  \n",
       "985577430d9c17e113d00a3923df0e4e6bcf452452fdd2e...      0.0  \n",
       "540ef136b0435594562ea3596c65104a42977047590f2ae...      0.0  \n",
       "\n",
       "[10002 rows x 897 columns]"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from __future__ import print_function\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from sklearn.cluster import KMeans\n",
    "import string\n",
    "import os\n",
    "\n",
    "from ipywidgets import interact, interactive, fixed, interact_manual\n",
    "import ipywidgets as widgets\n",
    "\n",
    "word_content_df = pd.read_csv(open(os.path.join(os.getcwd(), \"vba_features.csv\")),index_col=0)\n",
    "\n",
    "word_content_df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2. We now initialize our kmeans model. For our purposes we have it divide the data into 5 clusters, though in actual practice we divide it into up to 5000. We then fit our model to the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "word_content_kmeans = KMeans(n_clusters=5).fit(word_content_df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3. We are now able to get our data displayed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3bcee44080884e358edcbae159dee3a9",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "interactive(children=(IntSlider(value=396, description='xs', max=896), IntSlider(value=374, description='ys', …"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "<function __main__.slider_explore(xs=396, ys=374, zs=398)>"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "from mpl_toolkits.mplot3d import Axes3D\n",
    "\n",
    "def slider_explore(xs=396,ys=374,zs=398):\n",
    "    \n",
    "    fig = plt.figure(1, figsize=(10, 9))\n",
    "    ax = Axes3D(fig, rect=[0, 0, .95, 1], elev=48, azim=134)\n",
    "    labels = word_content_kmeans.labels_\n",
    "\n",
    "    ax.scatter(word_content_df.iloc[:,xs],word_content_df.iloc[:,ys],word_content_df.iloc[:,zs],\n",
    "               c=labels.astype(np.float))\n",
    "\n",
    "    ax.margins(x=0.00000001) \n",
    "\n",
    "    ax.w_xaxis.set_ticklabels([])\n",
    "    ax.w_yaxis.set_ticklabels([])\n",
    "    ax.w_zaxis.set_ticklabels([])\n",
    "    ax.set_xlabel(word_content_df.columns[xs])\n",
    "    ax.set_ylabel(word_content_df.columns[ys])\n",
    "    ax.set_zlabel(word_content_df.columns[zs])\n",
    "    ax.set_title('Kmeans Word Content')\n",
    "    ax.dist = 12\n",
    "\n",
    "interact(slider_explore,xs=(0,896,1),ys=(0,896,1),zs=(0,896,1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "4. In addition to these clusters, we are also able to get information about how malicious/benign each one is, based on how their members are classified in our database. To do this, we've included the hashes paired with their VirusTotal scores, with the higher the number the greater the chance of maliciousness, and our own manual labels, if available."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>vt_score</th>\n",
       "      <th>classification</th>\n",
       "      <th>abs</th>\n",
       "      <th>accelerator</th>\n",
       "      <th>activate</th>\n",
       "      <th>activecell</th>\n",
       "      <th>activecodepane</th>\n",
       "      <th>activecontrol</th>\n",
       "      <th>activedocument</th>\n",
       "      <th>activesheet</th>\n",
       "      <th>...</th>\n",
       "      <th>Join</th>\n",
       "      <th>LBound</th>\n",
       "      <th>Split</th>\n",
       "      <th>UBound</th>\n",
       "      <th>CurDir</th>\n",
       "      <th>Dir</th>\n",
       "      <th>FileAttr</th>\n",
       "      <th>FileDateTime</th>\n",
       "      <th>FileLen</th>\n",
       "      <th>GetAttr</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>00027b55ffe7329faff173bc3046f579d176c5a79091bf21f31062e17bfec922.macro</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.168157</td>\n",
       "      <td>0.058883</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.196722</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>00125528f276afcb74b5607e38b03edd41efafac58570589ef08b983cfa1231d.macro</th>\n",
       "      <td>14</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.013680</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>001331fc0a289089ddeaab9ece4b1cf919f4852afe42b6ad64e672e0afccc588.macro</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0014a559b4421bcc8f002e9a8b130f47ca04b7944ba89cf6e80524ed2912474c.macro</th>\n",
       "      <td>16</td>\n",
       "      <td>2</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.337177</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>002d9d8f1664df94a985beff2388badaca96ae46bafa92df76eb19d18c154dcd.macro</th>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.028296</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.013241</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ffd776fd6c53a5d4053d2f3c5e69aa513de519ebd9d915cc0cc1f6279c0e8326.macro</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0.005087</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.006297</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.515676</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ffde3b4dfa144508fb40ff4a57d1659f57e2a3432ed2aad7fe710fd60ed6e271.macro</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.687522</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ffe3c1134a7ca913889b1ce47dabd218bc45d4d8bedeb7fd8448b6cb05d93d84.macro</th>\n",
       "      <td>13</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fff7628f553c5df11fd36add8eaf80910bb80293fb1d01ec8cb79e03108de057.macro</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.058392</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.022760</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>fff950e8d9428d8415819e5e6fdd6667c843d9a6942a226f89d6a90c1e282c81.macro</th>\n",
       "      <td>11</td>\n",
       "      <td>2</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10000 rows × 899 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                    vt_score  classification  \\\n",
       "00027b55ffe7329faff173bc3046f579d176c5a79091bf2...         0               0   \n",
       "00125528f276afcb74b5607e38b03edd41efafac5857058...        14               0   \n",
       "001331fc0a289089ddeaab9ece4b1cf919f4852afe42b6a...         0               0   \n",
       "0014a559b4421bcc8f002e9a8b130f47ca04b7944ba89cf...        16               2   \n",
       "002d9d8f1664df94a985beff2388badaca96ae46bafa92d...         5               0   \n",
       "...                                                      ...             ...   \n",
       "ffd776fd6c53a5d4053d2f3c5e69aa513de519ebd9d915c...         1               1   \n",
       "ffde3b4dfa144508fb40ff4a57d1659f57e2a3432ed2aad...         0               0   \n",
       "ffe3c1134a7ca913889b1ce47dabd218bc45d4d8bedeb7f...        13               0   \n",
       "fff7628f553c5df11fd36add8eaf80910bb80293fb1d01e...         0               0   \n",
       "fff950e8d9428d8415819e5e6fdd6667c843d9a6942a226...        11               2   \n",
       "\n",
       "                                                         abs  accelerator  \\\n",
       "00027b55ffe7329faff173bc3046f579d176c5a79091bf2...  0.000000          0.0   \n",
       "00125528f276afcb74b5607e38b03edd41efafac5857058...  0.000000          0.0   \n",
       "001331fc0a289089ddeaab9ece4b1cf919f4852afe42b6a...  0.000000          0.0   \n",
       "0014a559b4421bcc8f002e9a8b130f47ca04b7944ba89cf...  0.000000          0.0   \n",
       "002d9d8f1664df94a985beff2388badaca96ae46bafa92d...  0.000000          0.0   \n",
       "...                                                      ...          ...   \n",
       "ffd776fd6c53a5d4053d2f3c5e69aa513de519ebd9d915c...  0.005087          0.0   \n",
       "ffde3b4dfa144508fb40ff4a57d1659f57e2a3432ed2aad...  0.000000          0.0   \n",
       "ffe3c1134a7ca913889b1ce47dabd218bc45d4d8bedeb7f...  0.000000          0.0   \n",
       "fff7628f553c5df11fd36add8eaf80910bb80293fb1d01e...  0.000000          0.0   \n",
       "fff950e8d9428d8415819e5e6fdd6667c843d9a6942a226...  0.000000          0.0   \n",
       "\n",
       "                                                    activate  activecell  \\\n",
       "00027b55ffe7329faff173bc3046f579d176c5a79091bf2...  0.168157    0.058883   \n",
       "00125528f276afcb74b5607e38b03edd41efafac5857058...  0.013680    0.000000   \n",
       "001331fc0a289089ddeaab9ece4b1cf919f4852afe42b6a...  0.000000    0.000000   \n",
       "0014a559b4421bcc8f002e9a8b130f47ca04b7944ba89cf...  0.000000    0.000000   \n",
       "002d9d8f1664df94a985beff2388badaca96ae46bafa92d...  0.028296    0.000000   \n",
       "...                                                      ...         ...   \n",
       "ffd776fd6c53a5d4053d2f3c5e69aa513de519ebd9d915c...  0.006297    0.000000   \n",
       "ffde3b4dfa144508fb40ff4a57d1659f57e2a3432ed2aad...  0.000000    0.000000   \n",
       "ffe3c1134a7ca913889b1ce47dabd218bc45d4d8bedeb7f...  0.000000    0.000000   \n",
       "fff7628f553c5df11fd36add8eaf80910bb80293fb1d01e...  0.000000    0.058392   \n",
       "fff950e8d9428d8415819e5e6fdd6667c843d9a6942a226...  0.000000    0.000000   \n",
       "\n",
       "                                                    activecodepane  \\\n",
       "00027b55ffe7329faff173bc3046f579d176c5a79091bf2...             0.0   \n",
       "00125528f276afcb74b5607e38b03edd41efafac5857058...             0.0   \n",
       "001331fc0a289089ddeaab9ece4b1cf919f4852afe42b6a...             0.0   \n",
       "0014a559b4421bcc8f002e9a8b130f47ca04b7944ba89cf...             0.0   \n",
       "002d9d8f1664df94a985beff2388badaca96ae46bafa92d...             0.0   \n",
       "...                                                            ...   \n",
       "ffd776fd6c53a5d4053d2f3c5e69aa513de519ebd9d915c...             0.0   \n",
       "ffde3b4dfa144508fb40ff4a57d1659f57e2a3432ed2aad...             0.0   \n",
       "ffe3c1134a7ca913889b1ce47dabd218bc45d4d8bedeb7f...             0.0   \n",
       "fff7628f553c5df11fd36add8eaf80910bb80293fb1d01e...             0.0   \n",
       "fff950e8d9428d8415819e5e6fdd6667c843d9a6942a226...             0.0   \n",
       "\n",
       "                                                    activecontrol  \\\n",
       "00027b55ffe7329faff173bc3046f579d176c5a79091bf2...            0.0   \n",
       "00125528f276afcb74b5607e38b03edd41efafac5857058...            0.0   \n",
       "001331fc0a289089ddeaab9ece4b1cf919f4852afe42b6a...            0.0   \n",
       "0014a559b4421bcc8f002e9a8b130f47ca04b7944ba89cf...            0.0   \n",
       "002d9d8f1664df94a985beff2388badaca96ae46bafa92d...            0.0   \n",
       "...                                                           ...   \n",
       "ffd776fd6c53a5d4053d2f3c5e69aa513de519ebd9d915c...            0.0   \n",
       "ffde3b4dfa144508fb40ff4a57d1659f57e2a3432ed2aad...            0.0   \n",
       "ffe3c1134a7ca913889b1ce47dabd218bc45d4d8bedeb7f...            0.0   \n",
       "fff7628f553c5df11fd36add8eaf80910bb80293fb1d01e...            0.0   \n",
       "fff950e8d9428d8415819e5e6fdd6667c843d9a6942a226...            0.0   \n",
       "\n",
       "                                                    activedocument  \\\n",
       "00027b55ffe7329faff173bc3046f579d176c5a79091bf2...        0.000000   \n",
       "00125528f276afcb74b5607e38b03edd41efafac5857058...        0.000000   \n",
       "001331fc0a289089ddeaab9ece4b1cf919f4852afe42b6a...        0.000000   \n",
       "0014a559b4421bcc8f002e9a8b130f47ca04b7944ba89cf...        0.337177   \n",
       "002d9d8f1664df94a985beff2388badaca96ae46bafa92d...        0.000000   \n",
       "...                                                            ...   \n",
       "ffd776fd6c53a5d4053d2f3c5e69aa513de519ebd9d915c...        0.000000   \n",
       "ffde3b4dfa144508fb40ff4a57d1659f57e2a3432ed2aad...        0.687522   \n",
       "ffe3c1134a7ca913889b1ce47dabd218bc45d4d8bedeb7f...        0.000000   \n",
       "fff7628f553c5df11fd36add8eaf80910bb80293fb1d01e...        0.000000   \n",
       "fff950e8d9428d8415819e5e6fdd6667c843d9a6942a226...        0.000000   \n",
       "\n",
       "                                                    activesheet  ...  Join  \\\n",
       "00027b55ffe7329faff173bc3046f579d176c5a79091bf2...     0.196722  ...   0.0   \n",
       "00125528f276afcb74b5607e38b03edd41efafac5857058...     0.000000  ...   0.0   \n",
       "001331fc0a289089ddeaab9ece4b1cf919f4852afe42b6a...     0.000000  ...   0.0   \n",
       "0014a559b4421bcc8f002e9a8b130f47ca04b7944ba89cf...     0.000000  ...   0.0   \n",
       "002d9d8f1664df94a985beff2388badaca96ae46bafa92d...     0.013241  ...   0.0   \n",
       "...                                                         ...  ...   ...   \n",
       "ffd776fd6c53a5d4053d2f3c5e69aa513de519ebd9d915c...     0.515676  ...   0.0   \n",
       "ffde3b4dfa144508fb40ff4a57d1659f57e2a3432ed2aad...     0.000000  ...   0.0   \n",
       "ffe3c1134a7ca913889b1ce47dabd218bc45d4d8bedeb7f...     0.000000  ...   0.0   \n",
       "fff7628f553c5df11fd36add8eaf80910bb80293fb1d01e...     0.022760  ...   0.0   \n",
       "fff950e8d9428d8415819e5e6fdd6667c843d9a6942a226...     0.000000  ...   0.0   \n",
       "\n",
       "                                                    LBound  Split  UBound  \\\n",
       "00027b55ffe7329faff173bc3046f579d176c5a79091bf2...     0.0    0.0     0.0   \n",
       "00125528f276afcb74b5607e38b03edd41efafac5857058...     0.0    0.0     0.0   \n",
       "001331fc0a289089ddeaab9ece4b1cf919f4852afe42b6a...     0.0    0.0     0.0   \n",
       "0014a559b4421bcc8f002e9a8b130f47ca04b7944ba89cf...     0.0    0.0     0.0   \n",
       "002d9d8f1664df94a985beff2388badaca96ae46bafa92d...     0.0    0.0     0.0   \n",
       "...                                                    ...    ...     ...   \n",
       "ffd776fd6c53a5d4053d2f3c5e69aa513de519ebd9d915c...     0.0    0.0     0.0   \n",
       "ffde3b4dfa144508fb40ff4a57d1659f57e2a3432ed2aad...     0.0    0.0     0.0   \n",
       "ffe3c1134a7ca913889b1ce47dabd218bc45d4d8bedeb7f...     0.0    0.0     0.0   \n",
       "fff7628f553c5df11fd36add8eaf80910bb80293fb1d01e...     0.0    0.0     0.0   \n",
       "fff950e8d9428d8415819e5e6fdd6667c843d9a6942a226...     0.0    0.0     0.0   \n",
       "\n",
       "                                                    CurDir  Dir  FileAttr  \\\n",
       "00027b55ffe7329faff173bc3046f579d176c5a79091bf2...     0.0  0.0       0.0   \n",
       "00125528f276afcb74b5607e38b03edd41efafac5857058...     0.0  0.0       0.0   \n",
       "001331fc0a289089ddeaab9ece4b1cf919f4852afe42b6a...     0.0  0.0       0.0   \n",
       "0014a559b4421bcc8f002e9a8b130f47ca04b7944ba89cf...     0.0  0.0       0.0   \n",
       "002d9d8f1664df94a985beff2388badaca96ae46bafa92d...     0.0  0.0       0.0   \n",
       "...                                                    ...  ...       ...   \n",
       "ffd776fd6c53a5d4053d2f3c5e69aa513de519ebd9d915c...     0.0  0.0       0.0   \n",
       "ffde3b4dfa144508fb40ff4a57d1659f57e2a3432ed2aad...     0.0  0.0       0.0   \n",
       "ffe3c1134a7ca913889b1ce47dabd218bc45d4d8bedeb7f...     0.0  0.0       0.0   \n",
       "fff7628f553c5df11fd36add8eaf80910bb80293fb1d01e...     0.0  0.0       0.0   \n",
       "fff950e8d9428d8415819e5e6fdd6667c843d9a6942a226...     0.0  0.0       0.0   \n",
       "\n",
       "                                                    FileDateTime  FileLen  \\\n",
       "00027b55ffe7329faff173bc3046f579d176c5a79091bf2...           0.0      0.0   \n",
       "00125528f276afcb74b5607e38b03edd41efafac5857058...           0.0      0.0   \n",
       "001331fc0a289089ddeaab9ece4b1cf919f4852afe42b6a...           0.0      0.0   \n",
       "0014a559b4421bcc8f002e9a8b130f47ca04b7944ba89cf...           0.0      0.0   \n",
       "002d9d8f1664df94a985beff2388badaca96ae46bafa92d...           0.0      0.0   \n",
       "...                                                          ...      ...   \n",
       "ffd776fd6c53a5d4053d2f3c5e69aa513de519ebd9d915c...           0.0      0.0   \n",
       "ffde3b4dfa144508fb40ff4a57d1659f57e2a3432ed2aad...           0.0      0.0   \n",
       "ffe3c1134a7ca913889b1ce47dabd218bc45d4d8bedeb7f...           0.0      0.0   \n",
       "fff7628f553c5df11fd36add8eaf80910bb80293fb1d01e...           0.0      0.0   \n",
       "fff950e8d9428d8415819e5e6fdd6667c843d9a6942a226...           0.0      0.0   \n",
       "\n",
       "                                                    GetAttr  \n",
       "00027b55ffe7329faff173bc3046f579d176c5a79091bf2...      0.0  \n",
       "00125528f276afcb74b5607e38b03edd41efafac5857058...      0.0  \n",
       "001331fc0a289089ddeaab9ece4b1cf919f4852afe42b6a...      0.0  \n",
       "0014a559b4421bcc8f002e9a8b130f47ca04b7944ba89cf...      0.0  \n",
       "002d9d8f1664df94a985beff2388badaca96ae46bafa92d...      0.0  \n",
       "...                                                     ...  \n",
       "ffd776fd6c53a5d4053d2f3c5e69aa513de519ebd9d915c...      0.0  \n",
       "ffde3b4dfa144508fb40ff4a57d1659f57e2a3432ed2aad...      0.0  \n",
       "ffe3c1134a7ca913889b1ce47dabd218bc45d4d8bedeb7f...      0.0  \n",
       "fff7628f553c5df11fd36add8eaf80910bb80293fb1d01e...      0.0  \n",
       "fff950e8d9428d8415819e5e6fdd6667c843d9a6942a226...      0.0  \n",
       "\n",
       "[10000 rows x 899 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "classification_df = pd.read_csv(open(os.path.join(os.getcwd(), 'classification.csv')),index_col = 0)\n",
    "df = classification_df.merge(word_content_df,left_index=True,right_index=True)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "6. The clusters can now be evaluated on their maliciousness/benignness, in a similar fashion to how they were displayed above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "83314eab776a440da97e3e65a68bf863",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "interactive(children=(IntSlider(value=398, description='xs', max=896), IntSlider(value=376, description='ys', …"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "<function __main__.slider_explore(xs=398, ys=376, zs=400)>"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def slider_explore(xs=398,ys=376,zs=400):\n",
    "    \n",
    "    fig = plt.figure(1, figsize=(10, 9))\n",
    "    ax = Axes3D(fig, rect=[0, 0, .95, 1], elev=48, azim=134)\n",
    "    labels = df['classification'].to_numpy()\n",
    "\n",
    "    ax.scatter(df.iloc[:,xs],df.iloc[:,ys],df.iloc[:,zs],\n",
    "               c=labels.astype(np.float))\n",
    "\n",
    "    ax.w_xaxis.set_ticklabels([])\n",
    "    ax.w_yaxis.set_ticklabels([])\n",
    "    ax.w_zaxis.set_ticklabels([])\n",
    "    ax.set_xlabel(df.columns[xs])\n",
    "    ax.set_ylabel(df.columns[ys])\n",
    "    ax.set_zlabel(df.columns[zs])\n",
    "    ax.set_title('Kmeans Word Content')\n",
    "    ax.dist = 12\n",
    "\n",
    "interact(slider_explore,xs=(0,896,1),ys=(0,896,1),zs=(0,896,1))\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}