data_prep1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We load three data sets that we want to merge into one big data set. `cpt` contains all waypoints of chimpanzee travel, one point recorded every 15 m on average. It also includes information on whether points can be considered change points. `wp` is a dataset that only contains the points where chimpanzees showed special activity such as feeding or resting. `tr` finally is a GeoJSON file that contains cleaned paths. Because the order and number of waypoints of these cleaned routes might differ from the ones contained in the other data sets, we preserve the structure of `tr` and add information that we are interested in from the other data sets. In the process, we also create segments that we will use for the modelling.\n",
    "\n",
    "We start with the `pt` data set and parse the datetime information:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>OBJECTID</th><th>xdate</th><th>prg</th><th>code</th><th>xname</th><th>xtime</th><th>focal</th><th>lat</th><th>lon</th></tr><tr><th></th><th>Int64</th><th>String</th><th>Int64</th><th>Missing</th><th>String</th><th>String</th><th>String</th><th>Float64</th><th>Float64</th></tr></thead><tbody><p>40,800 rows × 28 columns (omitted printing of 19 columns)</p><tr><th>1</th><td>1</td><td>2007/10/04</td><td>0</td><td><em>missing</em></td><td>s0</td><td>1899/12/30</td><td>T3</td><td>0.57843</td><td>30.3669</td></tr><tr><th>2</th><td>2</td><td>2007/10/04</td><td>1</td><td><em>missing</em></td><td>s0</td><td>1899/12/30</td><td>T3</td><td>0.57837</td><td>30.3671</td></tr><tr><th>3</th><td>91</td><td>2007/10/04</td><td>2</td><td><em>missing</em></td><td>s0</td><td>1899/12/30</td><td>T3</td><td>0.57818</td><td>30.3671</td></tr><tr><th>4</th><td>92</td><td>2007/10/04</td><td>3</td><td><em>missing</em></td><td>s0</td><td>1899/12/30</td><td>T3</td><td>0.57798</td><td>30.3671</td></tr><tr><th>5</th><td>93</td><td>2007/10/04</td><td>0</td><td><em>missing</em></td><td>s1</td><td>1899/12/30</td><td>A3</td><td>0.57778</td><td>30.3664</td></tr><tr><th>6</th><td>94</td><td>2007/10/04</td><td>1</td><td><em>missing</em></td><td>s1</td><td>1899/12/30</td><td>A3</td><td>0.57782</td><td>30.3668</td></tr><tr><th>7</th><td>95</td><td>2007/10/04</td><td>2</td><td><em>missing</em></td><td>s1</td><td>1899/12/30</td><td>A3</td><td>0.57783</td><td>30.3669</td></tr><tr><th>8</th><td>96</td><td>2007/10/04</td><td>3</td><td><em>missing</em></td><td>s1</td><td>1899/12/30</td><td>A3</td><td>0.57788</td><td>30.367</td></tr><tr><th>9</th><td>97</td><td>2007/10/04</td><td>4</td><td><em>missing</em></td><td>s1</td><td>1899/12/30</td><td>A3</td><td>0.57792</td><td>30.3671</td></tr><tr><th>10</th><td>98</td><td>2007/10/04</td><td>5</td><td><em>missing</em></td><td>s1</td><td>1899/12/30</td><td>A3</td><td>0.57793</td><td>30.3672</td></tr><tr><th>11</th><td>99</td><td>2007/10/04</td><td>6</td><td><em>missing</em></td><td>s1</td><td>1899/12/30</td><td>A3</td><td>0.57798</td><td>30.3673</td></tr><tr><th>12</th><td>100</td><td>2007/10/04</td><td>7</td><td><em>missing</em></td><td>s1</td><td>1899/12/30</td><td>A3</td><td>0.578</td><td>30.3674</td></tr><tr><th>13</th><td>101</td><td>2007/10/04</td><td>0</td><td><em>missing</em></td><td>s2</td><td>1899/12/30</td><td>GG</td><td>0.58947</td><td>30.3628</td></tr><tr><th>14</th><td>102</td><td>2007/10/04</td><td>1</td><td><em>missing</em></td><td>s2</td><td>1899/12/30</td><td>GG</td><td>0.58913</td><td>30.3629</td></tr><tr><th>15</th><td>103</td><td>2007/10/04</td><td>2</td><td><em>missing</em></td><td>s2</td><td>1899/12/30</td><td>GG</td><td>0.589</td><td>30.363</td></tr><tr><th>16</th><td>104</td><td>2007/10/04</td><td>3</td><td><em>missing</em></td><td>s2</td><td>1899/12/30</td><td>GG</td><td>0.58885</td><td>30.3631</td></tr><tr><th>17</th><td>105</td><td>2007/10/04</td><td>4</td><td><em>missing</em></td><td>s2</td><td>1899/12/30</td><td>GG</td><td>0.58873</td><td>30.3632</td></tr><tr><th>18</th><td>107</td><td>2007/10/04</td><td>5</td><td><em>missing</em></td><td>s2</td><td>1899/12/30</td><td>GG</td><td>0.58858</td><td>30.3634</td></tr><tr><th>19</th><td>108</td><td>2007/10/04</td><td>6</td><td><em>missing</em></td><td>s2</td><td>1899/12/30</td><td>GG</td><td>0.58847</td><td>30.3636</td></tr><tr><th>20</th><td>109</td><td>2007/10/04</td><td>7</td><td><em>missing</em></td><td>s2</td><td>1899/12/30</td><td>GG</td><td>0.5883</td><td>30.3637</td></tr><tr><th>21</th><td>8161</td><td>2007/10/04</td><td>8</td><td><em>missing</em></td><td>s2</td><td>1899/12/30</td><td>GG</td><td>0.58813</td><td>30.3639</td></tr><tr><th>22</th><td>8162</td><td>2007/10/04</td><td>9</td><td><em>missing</em></td><td>s2</td><td>1899/12/30</td><td>GG</td><td>0.5881</td><td>30.3641</td></tr><tr><th>23</th><td>8163</td><td>2007/10/04</td><td>10</td><td><em>missing</em></td><td>s2</td><td>1899/12/30</td><td>GG</td><td>0.58802</td><td>30.3642</td></tr><tr><th>24</th><td>8164</td><td>2007/10/04</td><td>11</td><td><em>missing</em></td><td>s2</td><td>1899/12/30</td><td>GG</td><td>0.58782</td><td>30.3643</td></tr><tr><th>25</th><td>8165</td><td>2007/10/04</td><td>12</td><td><em>missing</em></td><td>s2</td><td>1899/12/30</td><td>GG</td><td>0.58773</td><td>30.3644</td></tr><tr><th>26</th><td>8166</td><td>2007/10/04</td><td>13</td><td><em>missing</em></td><td>s2</td><td>1899/12/30</td><td>GG</td><td>0.58772</td><td>30.3645</td></tr><tr><th>27</th><td>8167</td><td>2007/10/04</td><td>14</td><td><em>missing</em></td><td>s2</td><td>1899/12/30</td><td>GG</td><td>0.58772</td><td>30.3646</td></tr><tr><th>28</th><td>8168</td><td>2007/10/04</td><td>15</td><td><em>missing</em></td><td>s2</td><td>1899/12/30</td><td>GG</td><td>0.58772</td><td>30.3648</td></tr><tr><th>29</th><td>8169</td><td>2007/10/04</td><td>16</td><td><em>missing</em></td><td>s2</td><td>1899/12/30</td><td>GG</td><td>0.58773</td><td>30.365</td></tr><tr><th>30</th><td>8170</td><td>2007/10/04</td><td>17</td><td><em>missing</em></td><td>s2</td><td>1899/12/30</td><td>GG</td><td>0.58768</td><td>30.3651</td></tr><tr><th>&vellip;</th><td>&vellip;</td><td>&vellip;</td><td>&vellip;</td><td>&vellip;</td><td>&vellip;</td><td>&vellip;</td><td>&vellip;</td><td>&vellip;</td><td>&vellip;</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccccccccc}\n",
       "\t& OBJECTID & xdate & prg & code & xname & xtime & focal & lat & lon & \\\\\n",
       "\t\\hline\n",
       "\t& Int64 & String & Int64 & Missing & String & String & String & Float64 & Float64 & \\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 2007/10/04 & 0 & \\emph{missing} & s0 & 1899/12/30 & T3 & 0.57843 & 30.3669 & $\\dots$ \\\\\n",
       "\t2 & 2 & 2007/10/04 & 1 & \\emph{missing} & s0 & 1899/12/30 & T3 & 0.57837 & 30.3671 & $\\dots$ \\\\\n",
       "\t3 & 91 & 2007/10/04 & 2 & \\emph{missing} & s0 & 1899/12/30 & T3 & 0.57818 & 30.3671 & $\\dots$ \\\\\n",
       "\t4 & 92 & 2007/10/04 & 3 & \\emph{missing} & s0 & 1899/12/30 & T3 & 0.57798 & 30.3671 & $\\dots$ \\\\\n",
       "\t5 & 93 & 2007/10/04 & 0 & \\emph{missing} & s1 & 1899/12/30 & A3 & 0.57778 & 30.3664 & $\\dots$ \\\\\n",
       "\t6 & 94 & 2007/10/04 & 1 & \\emph{missing} & s1 & 1899/12/30 & A3 & 0.57782 & 30.3668 & $\\dots$ \\\\\n",
       "\t7 & 95 & 2007/10/04 & 2 & \\emph{missing} & s1 & 1899/12/30 & A3 & 0.57783 & 30.3669 & $\\dots$ \\\\\n",
       "\t8 & 96 & 2007/10/04 & 3 & \\emph{missing} & s1 & 1899/12/30 & A3 & 0.57788 & 30.367 & $\\dots$ \\\\\n",
       "\t9 & 97 & 2007/10/04 & 4 & \\emph{missing} & s1 & 1899/12/30 & A3 & 0.57792 & 30.3671 & $\\dots$ \\\\\n",
       "\t10 & 98 & 2007/10/04 & 5 & \\emph{missing} & s1 & 1899/12/30 & A3 & 0.57793 & 30.3672 & $\\dots$ \\\\\n",
       "\t11 & 99 & 2007/10/04 & 6 & \\emph{missing} & s1 & 1899/12/30 & A3 & 0.57798 & 30.3673 & $\\dots$ \\\\\n",
       "\t12 & 100 & 2007/10/04 & 7 & \\emph{missing} & s1 & 1899/12/30 & A3 & 0.578 & 30.3674 & $\\dots$ \\\\\n",
       "\t13 & 101 & 2007/10/04 & 0 & \\emph{missing} & s2 & 1899/12/30 & GG & 0.58947 & 30.3628 & $\\dots$ \\\\\n",
       "\t14 & 102 & 2007/10/04 & 1 & \\emph{missing} & s2 & 1899/12/30 & GG & 0.58913 & 30.3629 & $\\dots$ \\\\\n",
       "\t15 & 103 & 2007/10/04 & 2 & \\emph{missing} & s2 & 1899/12/30 & GG & 0.589 & 30.363 & $\\dots$ \\\\\n",
       "\t16 & 104 & 2007/10/04 & 3 & \\emph{missing} & s2 & 1899/12/30 & GG & 0.58885 & 30.3631 & $\\dots$ \\\\\n",
       "\t17 & 105 & 2007/10/04 & 4 & \\emph{missing} & s2 & 1899/12/30 & GG & 0.58873 & 30.3632 & $\\dots$ \\\\\n",
       "\t18 & 107 & 2007/10/04 & 5 & \\emph{missing} & s2 & 1899/12/30 & GG & 0.58858 & 30.3634 & $\\dots$ \\\\\n",
       "\t19 & 108 & 2007/10/04 & 6 & \\emph{missing} & s2 & 1899/12/30 & GG & 0.58847 & 30.3636 & $\\dots$ \\\\\n",
       "\t20 & 109 & 2007/10/04 & 7 & \\emph{missing} & s2 & 1899/12/30 & GG & 0.5883 & 30.3637 & $\\dots$ \\\\\n",
       "\t21 & 8161 & 2007/10/04 & 8 & \\emph{missing} & s2 & 1899/12/30 & GG & 0.58813 & 30.3639 & $\\dots$ \\\\\n",
       "\t22 & 8162 & 2007/10/04 & 9 & \\emph{missing} & s2 & 1899/12/30 & GG & 0.5881 & 30.3641 & $\\dots$ \\\\\n",
       "\t23 & 8163 & 2007/10/04 & 10 & \\emph{missing} & s2 & 1899/12/30 & GG & 0.58802 & 30.3642 & $\\dots$ \\\\\n",
       "\t24 & 8164 & 2007/10/04 & 11 & \\emph{missing} & s2 & 1899/12/30 & GG & 0.58782 & 30.3643 & $\\dots$ \\\\\n",
       "\t25 & 8165 & 2007/10/04 & 12 & \\emph{missing} & s2 & 1899/12/30 & GG & 0.58773 & 30.3644 & $\\dots$ \\\\\n",
       "\t26 & 8166 & 2007/10/04 & 13 & \\emph{missing} & s2 & 1899/12/30 & GG & 0.58772 & 30.3645 & $\\dots$ \\\\\n",
       "\t27 & 8167 & 2007/10/04 & 14 & \\emph{missing} & s2 & 1899/12/30 & GG & 0.58772 & 30.3646 & $\\dots$ \\\\\n",
       "\t28 & 8168 & 2007/10/04 & 15 & \\emph{missing} & s2 & 1899/12/30 & GG & 0.58772 & 30.3648 & $\\dots$ \\\\\n",
       "\t29 & 8169 & 2007/10/04 & 16 & \\emph{missing} & s2 & 1899/12/30 & GG & 0.58773 & 30.365 & $\\dots$ \\\\\n",
       "\t30 & 8170 & 2007/10/04 & 17 & \\emph{missing} & s2 & 1899/12/30 & GG & 0.58768 & 30.3651 & $\\dots$ \\\\\n",
       "\t$\\dots$ & $\\dots$ & $\\dots$ & $\\dots$ & $\\dots$ & $\\dots$ & $\\dots$ & $\\dots$ & $\\dots$ & $\\dots$ &  \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "40800×28 DataFrame. Omitted printing of 22 columns\n",
       "│ Row   │ OBJECTID │ xdate      │ prg   │ code    │ xname  │ xtime      │\n",
       "│       │ \u001b[90mInt64\u001b[39m    │ \u001b[90mString\u001b[39m     │ \u001b[90mInt64\u001b[39m │ \u001b[90mMissing\u001b[39m │ \u001b[90mString\u001b[39m │ \u001b[90mString\u001b[39m     │\n",
       "├───────┼──────────┼────────────┼───────┼─────────┼────────┼────────────┤\n",
       "│ 1     │ 1        │ 2007/10/04 │ 0     │ \u001b[90mmissing\u001b[39m │ s0     │ 1899/12/30 │\n",
       "│ 2     │ 2        │ 2007/10/04 │ 1     │ \u001b[90mmissing\u001b[39m │ s0     │ 1899/12/30 │\n",
       "│ 3     │ 91       │ 2007/10/04 │ 2     │ \u001b[90mmissing\u001b[39m │ s0     │ 1899/12/30 │\n",
       "│ 4     │ 92       │ 2007/10/04 │ 3     │ \u001b[90mmissing\u001b[39m │ s0     │ 1899/12/30 │\n",
       "│ 5     │ 93       │ 2007/10/04 │ 0     │ \u001b[90mmissing\u001b[39m │ s1     │ 1899/12/30 │\n",
       "│ 6     │ 94       │ 2007/10/04 │ 1     │ \u001b[90mmissing\u001b[39m │ s1     │ 1899/12/30 │\n",
       "│ 7     │ 95       │ 2007/10/04 │ 2     │ \u001b[90mmissing\u001b[39m │ s1     │ 1899/12/30 │\n",
       "│ 8     │ 96       │ 2007/10/04 │ 3     │ \u001b[90mmissing\u001b[39m │ s1     │ 1899/12/30 │\n",
       "│ 9     │ 97       │ 2007/10/04 │ 4     │ \u001b[90mmissing\u001b[39m │ s1     │ 1899/12/30 │\n",
       "│ 10    │ 98       │ 2007/10/04 │ 5     │ \u001b[90mmissing\u001b[39m │ s1     │ 1899/12/30 │\n",
       "⋮\n",
       "│ 40790 │ 39889    │ 2009/05/11 │ 0     │ \u001b[90mmissing\u001b[39m │ s19    │ 1899/12/30 │\n",
       "│ 40791 │ 39890    │ 2009/05/11 │ 1     │ \u001b[90mmissing\u001b[39m │ s19    │ 1899/12/30 │\n",
       "│ 40792 │ 39891    │ 2009/05/11 │ 2     │ \u001b[90mmissing\u001b[39m │ s19    │ 1899/12/30 │\n",
       "│ 40793 │ 39892    │ 2009/05/11 │ 0     │ \u001b[90mmissing\u001b[39m │ s20    │ 1899/12/30 │\n",
       "│ 40794 │ 39893    │ 2009/05/11 │ 1     │ \u001b[90mmissing\u001b[39m │ s20    │ 1899/12/30 │\n",
       "│ 40795 │ 39894    │ 2009/05/11 │ 0     │ \u001b[90mmissing\u001b[39m │ s21    │ 1899/12/30 │\n",
       "│ 40796 │ 39895    │ 2009/05/11 │ 1     │ \u001b[90mmissing\u001b[39m │ s21    │ 1899/12/30 │\n",
       "│ 40797 │ 39896    │ 2009/05/11 │ 2     │ \u001b[90mmissing\u001b[39m │ s21    │ 1899/12/30 │\n",
       "│ 40798 │ 39897    │ 2009/05/11 │ 0     │ \u001b[90mmissing\u001b[39m │ s22    │ 1899/12/30 │\n",
       "│ 40799 │ 39898    │ 2009/05/11 │ 1     │ \u001b[90mmissing\u001b[39m │ s22    │ 1899/12/30 │\n",
       "│ 40800 │ 39899    │ 2009/05/11 │ 2     │ \u001b[90mmissing\u001b[39m │ s22    │ 1899/12/30 │"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "using DataFrames, CSV, Dates, JSON, Query\n",
    "\n",
    "cpt = CSV.read(\"data/cpt.csv\") |> DataFrame\n",
    "cpt.date = Date.(cpt.xdate, \"yyyy/mm/dd\")\n",
    "cpt.datetime = cpt.date .+ cpt.stime\n",
    "cpt.cpt = cpt.q5;\n",
    "sort!(cpt, (:datetime))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we deal with the `wp` data. We parse datetimes and create a column that holds information on the stationary time spent at each waypoint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "wp = CSV.read(\"data/wp.csv\") |> DataFrame\n",
    "wp.datetime = Date.(wp.xdate, \"dd/mm/yyyy\") .+ wp.xtIN\n",
    "wp.stattime = wp.xtOUT .- wp.xtIN; # stationary time at waypoint"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we dummy code the activity data contained in the `actcode` column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "wp.act = [1 for x in wp.actcode] # wp only includes points with activity\n",
    "wp.feeding = [x[2] == 'F' for x in wp.actcode]; "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After that, we merge both data sets into one big DataFrame `df` that contains all the relevant information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "# create df with all the points, and if point is waypoint, include additional info\n",
    "\n",
    "cpt_wp = leftjoin(cpt[:, [:focal, :prg, :datetime, :date, :lat, :lon, :tparty, :altitude, :cpt]], wp[:, [:datetime, :stattime, :act, :feeding]], on = :datetime);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "15×7 DataFrame\n",
      "│ Row │ focal  │ prg   │ lat     │ lon     │ cpt  │ act  │ feeding │\n",
      "│     │ \u001b[90mString\u001b[39m │ \u001b[90mInt64\u001b[39m │ \u001b[90mFloat64\u001b[39m │ \u001b[90mFloat64\u001b[39m │ \u001b[90mBool\u001b[39m │ \u001b[90mBool\u001b[39m │ \u001b[90mBool?\u001b[39m   │\n",
      "├─────┼────────┼───────┼─────────┼─────────┼──────┼──────┼─────────┤\n",
      "│ 1   │ T3     │ 0     │ 0.57843 │ 30.3669 │ 0    │ 0    │ 0       │\n",
      "│ 2   │ T3     │ 1     │ 0.57837 │ 30.3671 │ 0    │ 0    │ 0       │\n",
      "│ 3   │ T3     │ 2     │ 0.57818 │ 30.3671 │ 0    │ 0    │ 0       │\n",
      "│ 4   │ T3     │ 3     │ 0.57798 │ 30.3671 │ 0    │ 0    │ 0       │\n",
      "│ 5   │ A3     │ 0     │ 0.57778 │ 30.3664 │ 0    │ 0    │ 0       │\n",
      "│ 6   │ A3     │ 1     │ 0.57782 │ 30.3668 │ 0    │ 0    │ 0       │\n",
      "│ 7   │ A3     │ 2     │ 0.57783 │ 30.3669 │ 0    │ 0    │ 0       │\n",
      "│ 8   │ A3     │ 3     │ 0.57788 │ 30.367  │ 0    │ 0    │ 0       │\n",
      "│ 9   │ A3     │ 4     │ 0.57792 │ 30.3671 │ 0    │ 0    │ 0       │\n",
      "│ 10  │ A3     │ 5     │ 0.57793 │ 30.3672 │ 0    │ 0    │ 0       │\n",
      "│ 11  │ A3     │ 6     │ 0.57798 │ 30.3673 │ 0    │ 0    │ 0       │\n",
      "│ 12  │ A3     │ 7     │ 0.578   │ 30.3674 │ 0    │ 0    │ 0       │\n",
      "│ 13  │ GG     │ 0     │ 0.58947 │ 30.3628 │ 0    │ 0    │ 0       │\n",
      "│ 14  │ GG     │ 1     │ 0.58913 │ 30.3629 │ 0    │ 0    │ 0       │\n",
      "│ 15  │ GG     │ 2     │ 0.589   │ 30.363  │ 0    │ 0    │ 0       │"
     ]
    }
   ],
   "source": [
    "replace!(cpt_wp.act, missing => 0)\n",
    "replace!(cpt_wp.cpt, missing => 0)\n",
    "replace!(cpt_wp.feeding, missing => 0)\n",
    "cpt_wp.act = Bool.(cpt_wp.act) # int => bool\n",
    "cpt_wp.cpt = Bool.(cpt_wp.cpt)\n",
    "\n",
    "mask = vcat([false], [cpt_wp.lat[i] == cpt_wp.lat[i + 1] && cpt_wp.lon[i] == cpt_wp.lon[i + 1] for i in 1:nrow(cpt_wp)-1])\n",
    "cpt_wp = cpt_wp[map(!, mask), :] # remove duplicates\n",
    "\n",
    "print(cpt_wp[1:15, [:focal, :prg, :lat, :lon, :cpt, :act, :feeding]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are interested in travel that is potentially intentional. For this reason, we want to compartmentalise the daily travel into segments from one point of activity to another point of activity where we suspect, based on the Change Point Test, that they are ultimate destinations of travel (not just a proximate beacon). The function `count` iterates over the points and advances the segment count by one after the co-occurence of a change point and recorded activity. The presence of recorded activity is a necessary condition to be sure that the change point does not appear because of topological features (road system/ridge/river that requires a turn). Two data sets are created: A considers a change point if its has recorded activity of any sort, F only if it has feeding activity.\n",
    "\n",
    "We use the function `count` to create a column `segment` which takes the form `<focal individual>_<date>_<count>`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"data/points_f.csv\""
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mutable struct Counter\n",
    "    i::Int\n",
    "    num::Int \n",
    "end \n",
    "counter1 = Counter(1,1)\n",
    "counter2 = Counter(1,1)\n",
    "\n",
    "function count(df::DataFrame, counter::Counter, i::Int, col::Symbol, indices::Array{Int64,1})\n",
    "    if df.focal[counter.i] != df.focal[i] || Date(df.datetime[counter.i]) != Date(df.datetime[i])\n",
    "        counter.num = 1\n",
    "    end\n",
    "    counter.i = i\n",
    "    ret = counter.num\n",
    "    mini(i) = i - 2 < 1 ? i : i - 2\n",
    "    maxi(i) = i + 2 > nrow(df) ? i : i + 2 \n",
    "    if df[col][i] && (df.focal[mini(i)] == df.focal[maxi(i)] && Date(df.datetime[mini(i)]) == Date(df.datetime[maxi(i)]) ? \n",
    "            any(df.cpt[mini(i):maxi(i)]) : \n",
    "            df.cpt[i]) # we also check up to two points before and after the change point (if it's the same focal)\n",
    "        counter.num += 1\n",
    "        push!(indices, i)\n",
    "    end\n",
    "    return ret\n",
    "end\n",
    "\n",
    "indices_a = Int64[]\n",
    "indices_f = Int64[]\n",
    "df_a = cpt_wp\n",
    "df_f = copy(cpt_wp)\n",
    "df_a.segment = [df_a.focal[i] * \"_\" * string(Date(df_a.datetime[i])) * \"_\" * string(count(df_a, counter1, i, :act, indices_a)) for i in 1:nrow(df_a)] # create column \"segment\" (all activity considered)\n",
    "df_f.segment = [df_f.focal[i] * \"_\" * string(Date(df_f.datetime[i])) * \"_\" * string(count(df_f, counter2, i, :feeding, indices_f)) for i in 1:nrow(df_f)] # only feeding considered\n",
    "\n",
    "df_a_new = DataFrame(:segment => String[], :focal => String[], :datetime => DateTime[], :lat => Float64[], :lon => Float64[], :tparty => Union{Missing, String}[], :altitude => Int64[])\n",
    "df_f_new = DataFrame(:segment => String[], :focal => String[], :datetime => DateTime[], :lat => Float64[], :lon => Float64[], :tparty => Union{Missing, String}[], :altitude => Int64[])\n",
    "\n",
    "#print(filter(r -> r.focal == \"ST\" && Date(r.datetime) == Date(2007,10,13), cpt_wp)[:, [:lat, :lon, :cpt, :act, :feeding, :focal, :datetime]])\n",
    "#print(filter(r -> r.focal == \"ST\" && Date(r.datetime) == Date(2007,10,13), df_a)[:, [:segment, :lat, :lon]])\n",
    "#print(filter(r -> r.focal == \"ST\" && Date(r.datetime) == Date(2007,10,13), df_f)[:, [:segment, :lat, :lon]])\n",
    "\n",
    "for i in 1:nrow(df_a)\n",
    "    row = df_a[i, [:segment, :focal, :datetime, :lat, :lon, :tparty, :altitude]]\n",
    "    push!(df_a_new, row)\n",
    "    if in(i, indices_a) && i < nrow(df_a)\n",
    "        row.segment = df_a[i + 1, :segment]\n",
    "        row.focal = df_a[i + 1, :focal]\n",
    "        row.tparty = df_a[i + 1, :tparty]\n",
    "        push!(df_a_new, row)\n",
    "    end\n",
    "end\n",
    "\n",
    "for i in 1:nrow(df_f)\n",
    "    row = df_f[i, [:segment, :focal, :datetime, :lat, :lon, :tparty, :altitude]]\n",
    "    push!(df_f_new, row)\n",
    "    if in(i, indices_f) && i < nrow(df_f)\n",
    "        row.segment = df_f[i + 1, :segment]\n",
    "        row.focal = df_f[i + 1, :focal]\n",
    "        row.tparty = df_f[i + 1, :tparty]\n",
    "        push!(df_f_new, row)\n",
    "    end\n",
    "end\n",
    "\n",
    "CSV.write(\"data/points_a.csv\", df_a_new)\n",
    "CSV.write(\"data/points_f.csv\", df_f_new)"
   ]
  }
 ],
 "metadata": {
  "@webio": {
   "lastCommId": null,
   "lastKernelId": null
  },
  "kernelspec": {
   "display_name": "Julia 1.5.0",
   "language": "julia",
   "name": "julia-1.5"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.5.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}