5 高级数据管理.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**本章内容**\n",
    "\n",
    "    数学和统计函数\n",
    "    字符处理函数\n",
    "    循环和条件执行\n",
    "    自编函数\n",
    "    数据整合与重塑"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5.1 一个数据处理难题"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5.2 数值和字符处理函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.2.1 数学函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![数学函数.jpg](./image/数学函数.jpg)\n",
    "![数学函数2.jpg](./image/数学函数2.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.2.2 统计函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![统计函数.jpg](./image/统计函数.jpg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Your code contains a unicode char which cannot be displayed in your\n",
      "current locale and R will silently convert it to an escaped form when the\n",
      "R kernel executes this code. This can lead to subtle errors if you use\n",
      "such chars to do comparisons. For more information, please see\n",
      "https://github.com/IRkernel/repr/wiki/Problems-with-unicode-on-windows"
     ]
    },
    {
     "data": {
      "text/html": [
       "4.5"
      ],
      "text/latex": [
       "4.5"
      ],
      "text/markdown": [
       "4.5"
      ],
      "text/plain": [
       "[1] 4.5"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "2.44948974278318"
      ],
      "text/latex": [
       "2.44948974278318"
      ],
      "text/markdown": [
       "2.44948974278318"
      ],
      "text/plain": [
       "[1] 2.44949"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "4.5"
      ],
      "text/latex": [
       "4.5"
      ],
      "text/markdown": [
       "4.5"
      ],
      "text/plain": [
       "[1] 4.5"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "2.44948974278318"
      ],
      "text/latex": [
       "2.44948974278318"
      ],
      "text/markdown": [
       "2.44948974278318"
      ],
      "text/plain": [
       "[1] 2.44949"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#均值和标准差的计算\n",
    "x <- c(1,2,3,4,5,6,7,8)\n",
    "\n",
    "#简洁的方式\n",
    "mean(x)\n",
    "sd(x)\n",
    "\n",
    "#冗长的方式\n",
    "n <- length(x)\n",
    "meanx <- sum(x/n)\n",
    "css <- sum((x - meanx)^2 )\n",
    "sdx <- sqrt(css / (n-1))\n",
    "meanx\n",
    "sdx"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![数据的标准化.jpg](./image/数据的标准化.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.2.3 概率函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "概率函数用来生成特征已知的模拟数据，以及在用户编写的统计函数中计算概率值。\n",
    "\n",
    "在R中，概率函数形如 ：\n",
    "    \n",
    "    [dpqr]distribution_abbreviation()\n",
    "    \n",
    "其中第一个字母表示其所指分布的某一方面\n",
    "       \n",
    "    d = 密度函数（density）\n",
    "    p = 分布函数（distribution function）\n",
    "    q = 分位数函数（quantile function）\n",
    "    r = 生成随机数（随机偏差）\n",
    "    \n",
    "![概率分布](./image/概率分布.jpg)\n",
    "![正态函数.png](./image/正态函数.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. 设定随机数种子"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在每次生成伪随机数的时候，函数都会使用一个不同的种子，因此也会产生不同的结果。你\n",
    "可以通过函数set.seed()显式指定这个种子，让结果可以重现（reproducible）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<ol class=list-inline>\n",
       "\t<li>0.640310605289415</li>\n",
       "\t<li>0.0094957563560456</li>\n",
       "\t<li>0.232550506014377</li>\n",
       "\t<li>0.666083758231252</li>\n",
       "\t<li>0.514251141343266</li>\n",
       "</ol>\n"
      ],
      "text/latex": [
       "\\begin{enumerate*}\n",
       "\\item 0.640310605289415\n",
       "\\item 0.0094957563560456\n",
       "\\item 0.232550506014377\n",
       "\\item 0.666083758231252\n",
       "\\item 0.514251141343266\n",
       "\\end{enumerate*}\n"
      ],
      "text/markdown": [
       "1. 0.640310605289415\n",
       "2. 0.0094957563560456\n",
       "3. 0.232550506014377\n",
       "4. 0.666083758231252\n",
       "5. 0.514251141343266\n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "[1] 0.640310605 0.009495756 0.232550506 0.666083758 0.514251141"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<ol class=list-inline>\n",
       "\t<li>0.693591291783378</li>\n",
       "\t<li>0.544974835589528</li>\n",
       "\t<li>0.282733583590016</li>\n",
       "\t<li>0.923433484276757</li>\n",
       "\t<li>0.292315840255469</li>\n",
       "</ol>\n"
      ],
      "text/latex": [
       "\\begin{enumerate*}\n",
       "\\item 0.693591291783378\n",
       "\\item 0.544974835589528\n",
       "\\item 0.282733583590016\n",
       "\\item 0.923433484276757\n",
       "\\item 0.292315840255469\n",
       "\\end{enumerate*}\n"
      ],
      "text/markdown": [
       "1. 0.693591291783378\n",
       "2. 0.544974835589528\n",
       "3. 0.282733583590016\n",
       "4. 0.923433484276757\n",
       "5. 0.292315840255469\n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "[1] 0.6935913 0.5449748 0.2827336 0.9234335 0.2923158"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<ol class=list-inline>\n",
       "\t<li>0.113703411305323</li>\n",
       "\t<li>0.622299404814839</li>\n",
       "\t<li>0.609274732880294</li>\n",
       "\t<li>0.623379441676661</li>\n",
       "\t<li>0.860915383556858</li>\n",
       "</ol>\n"
      ],
      "text/latex": [
       "\\begin{enumerate*}\n",
       "\\item 0.113703411305323\n",
       "\\item 0.622299404814839\n",
       "\\item 0.609274732880294\n",
       "\\item 0.623379441676661\n",
       "\\item 0.860915383556858\n",
       "\\end{enumerate*}\n"
      ],
      "text/markdown": [
       "1. 0.113703411305323\n",
       "2. 0.622299404814839\n",
       "3. 0.609274732880294\n",
       "4. 0.623379441676661\n",
       "5. 0.860915383556858\n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "[1] 0.1137034 0.6222994 0.6092747 0.6233794 0.8609154"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<ol class=list-inline>\n",
       "\t<li>0.113703411305323</li>\n",
       "\t<li>0.622299404814839</li>\n",
       "\t<li>0.609274732880294</li>\n",
       "\t<li>0.623379441676661</li>\n",
       "\t<li>0.860915383556858</li>\n",
       "</ol>\n"
      ],
      "text/latex": [
       "\\begin{enumerate*}\n",
       "\\item 0.113703411305323\n",
       "\\item 0.622299404814839\n",
       "\\item 0.609274732880294\n",
       "\\item 0.623379441676661\n",
       "\\item 0.860915383556858\n",
       "\\end{enumerate*}\n"
      ],
      "text/markdown": [
       "1. 0.113703411305323\n",
       "2. 0.622299404814839\n",
       "3. 0.609274732880294\n",
       "4. 0.623379441676661\n",
       "5. 0.860915383556858\n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "[1] 0.1137034 0.6222994 0.6092747 0.6233794 0.8609154"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#生成服从正态分布的伪随机数\n",
    "\n",
    "runif(5)  #函数runif()用来生成0到1区间上服从均匀分布的伪随机数\n",
    "runif(5)\n",
    "set.seed(1234)  #通过函数set.seed()显式指定这个种子，让结果可以重现（reproducible）\n",
    "runif(5)\n",
    "set.seed(1234)\n",
    "runif(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. 生成多元正态数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在模拟研究和蒙特卡洛方法中，你经常需要获取来自给定均值向量和协方差阵的多元正态分布的数据。\n",
    "\n",
    "MASS包中的mvrnorm()函数可以让这个问题变得很容易。其调用格式为：\n",
    "\n",
    "mvrnorm(n, mean, sigma)\n",
    "\n",
    "n是你想要的样本大小，mean为均值向量，而sigma是方差协方差矩阵（或相关矩阵）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Your code contains a unicode char which cannot be displayed in your\n",
      "current locale and R will silently convert it to an escaped form when the\n",
      "R kernel executes this code. This can lead to subtle errors if you use\n",
      "such chars to do comparisons. For more information, please see\n",
      "https://github.com/IRkernel/repr/wiki/Problems-with-unicode-on-windows"
     ]
    },
    {
     "data": {
      "text/html": [
       "<ol class=list-inline>\n",
       "\t<li>500</li>\n",
       "\t<li>3</li>\n",
       "</ol>\n"
      ],
      "text/latex": [
       "\\begin{enumerate*}\n",
       "\\item 500\n",
       "\\item 3\n",
       "\\end{enumerate*}\n"
      ],
      "text/markdown": [
       "1. 500\n",
       "2. 3\n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "[1] 500   3"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead><tr><th scope=col>y</th><th scope=col>x1</th><th scope=col>x2</th></tr></thead>\n",
       "<tbody>\n",
       "\t<tr><td> 98.8</td><td> 41.3</td><td>3.43 </td></tr>\n",
       "\t<tr><td>244.5</td><td>205.2</td><td>3.80 </td></tr>\n",
       "\t<tr><td>375.7</td><td>186.7</td><td>2.51 </td></tr>\n",
       "\t<tr><td>-59.2</td><td> 11.2</td><td>4.71 </td></tr>\n",
       "\t<tr><td>313.0</td><td>111.0</td><td>3.45 </td></tr>\n",
       "\t<tr><td>288.8</td><td>185.1</td><td>2.72 </td></tr>\n",
       "\t<tr><td>134.8</td><td>165.0</td><td>4.39 </td></tr>\n",
       "\t<tr><td>171.7</td><td> 97.4</td><td>3.64 </td></tr>\n",
       "\t<tr><td>167.2</td><td>101.0</td><td>3.50 </td></tr>\n",
       "\t<tr><td>121.1</td><td> 94.5</td><td>4.10 </td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{r|lll}\n",
       " y & x1 & x2\\\\\n",
       "\\hline\n",
       "\t  98.8 &  41.3 & 3.43 \\\\\n",
       "\t 244.5 & 205.2 & 3.80 \\\\\n",
       "\t 375.7 & 186.7 & 2.51 \\\\\n",
       "\t -59.2 &  11.2 & 4.71 \\\\\n",
       "\t 313.0 & 111.0 & 3.45 \\\\\n",
       "\t 288.8 & 185.1 & 2.72 \\\\\n",
       "\t 134.8 & 165.0 & 4.39 \\\\\n",
       "\t 171.7 &  97.4 & 3.64 \\\\\n",
       "\t 167.2 & 101.0 & 3.50 \\\\\n",
       "\t 121.1 &  94.5 & 4.10 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "| y | x1 | x2 |\n",
       "|---|---|---|\n",
       "|  98.8 |  41.3 | 3.43  |\n",
       "| 244.5 | 205.2 | 3.80  |\n",
       "| 375.7 | 186.7 | 2.51  |\n",
       "| -59.2 |  11.2 | 4.71  |\n",
       "| 313.0 | 111.0 | 3.45  |\n",
       "| 288.8 | 185.1 | 2.72  |\n",
       "| 134.8 | 165.0 | 4.39  |\n",
       "| 171.7 |  97.4 | 3.64  |\n",
       "| 167.2 | 101.0 | 3.50  |\n",
       "| 121.1 |  94.5 | 4.10  |\n",
       "\n"
      ],
      "text/plain": [
       "   y     x1    x2  \n",
       "1   98.8  41.3 3.43\n",
       "2  244.5 205.2 3.80\n",
       "3  375.7 186.7 2.51\n",
       "4  -59.2  11.2 4.71\n",
       "5  313.0 111.0 3.45\n",
       "6  288.8 185.1 2.72\n",
       "7  134.8 165.0 4.39\n",
       "8  171.7  97.4 3.64\n",
       "9  167.2 101.0 3.50\n",
       "10 121.1  94.5 4.10"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#生成服从多元正态分布的数据\n",
    "\n",
    "#设定随机数种子\n",
    "library(MASS)\n",
    "options(digits=3)\n",
    "set.seed(1234)\n",
    "\n",
    "#指定均值向量、协方差阵\n",
    "mean <- c(230.7, 146.7, 3.6)  \n",
    "sigma <- matrix(c(15360.8, 6721.2, -47.1,\n",
    "                  6721.2, 4700.9, -16.5,\n",
    "                  -47.1, -16.5, 0.3), nrow=3, ncol=3)\n",
    "\n",
    "#生成数据\n",
    "mydata <- mvrnorm(500, mean, sigma)\n",
    "mydata <- as.data.frame(mydata)\n",
    "names(mydata) <- c(\"y\",\"x1\",\"x2\")\n",
    "\n",
    "#查看结果\n",
    "dim(mydata)\n",
    "head(mydata, n=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.2.4 字符处理函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "数学和统计函数是用来处理数值型数据的，而字符处理函数可以从文本型数据中抽取信息，或者为打印输出和生成报告重设文本的格式。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![字符处理函数.png](./image/字符处理函数.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "请注意，函数grep()、sub()和strsplit()能够搜索某个文本字符串（fixed=TRUE）或某个正则表达式（fixed=FALSE，默认值为FALSE）。正则表达式为文本模式的匹配提供了一套清晰而简练的语法。例如，正则表达式：\n",
    "\n",
    "^[hc]?at\n",
    "\n",
    "可匹配任意以0个或1个h或c开头、后接at的字符串。因此，此表达式可以匹配hat、cat和at，但不会匹配bat"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.2.5 其他实用函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![其他实用函数.jpg](./image/其他实用函数.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.2.6 将函数应用于矩阵和数据框"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "2.23606797749979"
      ],
      "text/latex": [
       "2.23606797749979"
      ],
      "text/markdown": [
       "2.23606797749979"
      ],
      "text/plain": [
       "[1] 2.236068"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "a <- 5\n",
    "sqrt(a)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<ol class=list-inline>\n",
       "\t<li>1</li>\n",
       "\t<li>6</li>\n",
       "\t<li>3</li>\n",
       "</ol>\n"
      ],
      "text/latex": [
       "\\begin{enumerate*}\n",
       "\\item 1\n",
       "\\item 6\n",
       "\\item 3\n",
       "\\end{enumerate*}\n"
      ],
      "text/markdown": [
       "1. 1\n",
       "2. 6\n",
       "3. 3\n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "[1] 1 6 3"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "b <- c(1.243, 5.654, 2.99)\n",
    "round(b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<tbody>\n",
       "\t<tr><td>0.2401800</td><td>0.8612100</td><td>0.4297635</td><td>0.6598139</td></tr>\n",
       "\t<tr><td>0.2199789</td><td>0.9544906</td><td>0.3065218</td><td>0.8959072</td></tr>\n",
       "\t<tr><td>0.3555674</td><td>0.5063445</td><td>0.4496468</td><td>0.2061033</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{llll}\n",
       "\t 0.2401800 & 0.8612100 & 0.4297635 & 0.6598139\\\\\n",
       "\t 0.2199789 & 0.9544906 & 0.3065218 & 0.8959072\\\\\n",
       "\t 0.3555674 & 0.5063445 & 0.4496468 & 0.2061033\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "| 0.2401800 | 0.8612100 | 0.4297635 | 0.6598139 | \n",
       "| 0.2199789 | 0.9544906 | 0.3065218 | 0.8959072 | \n",
       "| 0.3555674 | 0.5063445 | 0.4496468 | 0.2061033 | \n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "     [,1]      [,2]      [,3]      [,4]     \n",
       "[1,] 0.2401800 0.8612100 0.4297635 0.6598139\n",
       "[2,] 0.2199789 0.9544906 0.3065218 0.8959072\n",
       "[3,] 0.3555674 0.5063445 0.4496468 0.2061033"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "c <- matrix(runif(12),nrow=3)\n",
    "c"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<tbody>\n",
       "\t<tr><td>-1.426367  </td><td>-0.14941688</td><td>-0.8445203 </td><td>-0.4157974 </td></tr>\n",
       "\t<tr><td>-1.514224  </td><td>-0.04657748</td><td>-1.1824664 </td><td>-0.1099184 </td></tr>\n",
       "\t<tr><td>-1.034040  </td><td>-0.68053797</td><td>-0.7992929 </td><td>-1.5793776 </td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{llll}\n",
       "\t -1.426367   & -0.14941688 & -0.8445203  & -0.4157974 \\\\\n",
       "\t -1.514224   & -0.04657748 & -1.1824664  & -0.1099184 \\\\\n",
       "\t -1.034040   & -0.68053797 & -0.7992929  & -1.5793776 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "| -1.426367   | -0.14941688 | -0.8445203  | -0.4157974  | \n",
       "| -1.514224   | -0.04657748 | -1.1824664  | -0.1099184  | \n",
       "| -1.034040   | -0.68053797 | -0.7992929  | -1.5793776  | \n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "     [,1]      [,2]        [,3]       [,4]      \n",
       "[1,] -1.426367 -0.14941688 -0.8445203 -0.4157974\n",
       "[2,] -1.514224 -0.04657748 -1.1824664 -0.1099184\n",
       "[3,] -1.034040 -0.68053797 -0.7992929 -1.5793776"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "log(c)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "0.507127335586119"
      ],
      "text/latex": [
       "0.507127335586119"
      ],
      "text/markdown": [
       "0.507127335586119"
      ],
      "text/plain": [
       "[1] 0.5071273"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "mean(c)  #这里求得是矩阵中全部12个元素的均值"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "R中提供了一个apply()函数，可将一个任意函数“应用”到矩阵、数组、数据框的任何维\n",
    "度上。apply()函数的使用格式为：\n",
    "\n",
    "apply(x, MARGIN, FUN, ...)\n",
    "\n",
    "x为数据对象，MARGIN是维度的下标，FUN是由你指定的函数，而...则包括了任何想传递给FUN的参数。在矩阵或数据框中，MARGIN=1表示行，MARGIN=2表示列"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<tbody>\n",
       "\t<tr><td>-1.1929325  </td><td> 2.06834012 </td><td> 0.5828052  </td><td> 0.0462830  </td><td> 1.396451602</td></tr>\n",
       "\t<tr><td>-0.8901659  </td><td> 0.26522780 </td><td> 0.9717250  </td><td> 1.2989760  </td><td> 0.246375797</td></tr>\n",
       "\t<tr><td>-1.1477740  </td><td> 0.74370995 </td><td> 0.5889367  </td><td> 1.7164145  </td><td> 0.816634541</td></tr>\n",
       "\t<tr><td> 0.5858595  </td><td> 0.15373940 </td><td> 0.4904735  </td><td> 0.4711747  </td><td> 2.685357458</td></tr>\n",
       "\t<tr><td>-0.4405302  </td><td> 0.02303565 </td><td> 1.5235336  </td><td>-0.6414648  </td><td> 0.003724815</td></tr>\n",
       "\t<tr><td>-0.8849150  </td><td>-0.03116790 </td><td>-0.7894051  </td><td>-1.2186322  </td><td>-0.164809811</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{lllll}\n",
       "\t -1.1929325   &  2.06834012  &  0.5828052   &  0.0462830   &  1.396451602\\\\\n",
       "\t -0.8901659   &  0.26522780  &  0.9717250   &  1.2989760   &  0.246375797\\\\\n",
       "\t -1.1477740   &  0.74370995  &  0.5889367   &  1.7164145   &  0.816634541\\\\\n",
       "\t  0.5858595   &  0.15373940  &  0.4904735   &  0.4711747   &  2.685357458\\\\\n",
       "\t -0.4405302   &  0.02303565  &  1.5235336   & -0.6414648   &  0.003724815\\\\\n",
       "\t -0.8849150   & -0.03116790  & -0.7894051   & -1.2186322   & -0.164809811\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "| -1.1929325   |  2.06834012  |  0.5828052   |  0.0462830   |  1.396451602 | \n",
       "| -0.8901659   |  0.26522780  |  0.9717250   |  1.2989760   |  0.246375797 | \n",
       "| -1.1477740   |  0.74370995  |  0.5889367   |  1.7164145   |  0.816634541 | \n",
       "|  0.5858595   |  0.15373940  |  0.4904735   |  0.4711747   |  2.685357458 | \n",
       "| -0.4405302   |  0.02303565  |  1.5235336   | -0.6414648   |  0.003724815 | \n",
       "| -0.8849150   | -0.03116790  | -0.7894051   | -1.2186322   | -0.164809811 | \n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "     [,1]       [,2]        [,3]       [,4]       [,5]        \n",
       "[1,] -1.1929325  2.06834012  0.5828052  0.0462830  1.396451602\n",
       "[2,] -0.8901659  0.26522780  0.9717250  1.2989760  0.246375797\n",
       "[3,] -1.1477740  0.74370995  0.5889367  1.7164145  0.816634541\n",
       "[4,]  0.5858595  0.15373940  0.4904735  0.4711747  2.685357458\n",
       "[5,] -0.4405302  0.02303565  1.5235336 -0.6414648  0.003724815\n",
       "[6,] -0.8849150 -0.03116790 -0.7894051 -1.2186322 -0.164809811"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#将一个函数应用到矩阵的所有行（列）\n",
    "mydata <- matrix(rnorm(30), nrow=6)\n",
    "mydata"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<ol class=list-inline>\n",
       "\t<li>0.580189488894865</li>\n",
       "\t<li>0.378427744672679</li>\n",
       "\t<li>0.543584339322651</li>\n",
       "\t<li>0.877320902339905</li>\n",
       "\t<li>0.0936598106518473</li>\n",
       "\t<li>-0.617785995901677</li>\n",
       "</ol>\n"
      ],
      "text/latex": [
       "\\begin{enumerate*}\n",
       "\\item 0.580189488894865\n",
       "\\item 0.378427744672679\n",
       "\\item 0.543584339322651\n",
       "\\item 0.877320902339905\n",
       "\\item 0.0936598106518473\n",
       "\\item -0.617785995901677\n",
       "\\end{enumerate*}\n"
      ],
      "text/markdown": [
       "1. 0.580189488894865\n",
       "2. 0.378427744672679\n",
       "3. 0.543584339322651\n",
       "4. 0.877320902339905\n",
       "5. 0.0936598106518473\n",
       "6. -0.617785995901677\n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "[1]  0.58018949  0.37842774  0.54358434  0.87732090  0.09365981 -0.61778600"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "apply(mydata,1,mean) #计算每行的均值"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<ol class=list-inline>\n",
       "\t<li>-0.661743044829283</li>\n",
       "\t<li>0.537147504318957</li>\n",
       "\t<li>0.561344845515841</li>\n",
       "\t<li>0.278791869829964</li>\n",
       "\t<li>0.83062240014808</li>\n",
       "</ol>\n"
      ],
      "text/latex": [
       "\\begin{enumerate*}\n",
       "\\item -0.661743044829283\n",
       "\\item 0.537147504318957\n",
       "\\item 0.561344845515841\n",
       "\\item 0.278791869829964\n",
       "\\item 0.83062240014808\n",
       "\\end{enumerate*}\n"
      ],
      "text/markdown": [
       "1. -0.661743044829283\n",
       "2. 0.537147504318957\n",
       "3. 0.561344845515841\n",
       "4. 0.278791869829964\n",
       "5. 0.83062240014808\n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "[1] -0.6617430  0.5371475  0.5613448  0.2787919  0.8306224"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "apply(mydata,2,mean) #计算每列的均值"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<ol class=list-inline>\n",
       "\t<li>-0.840846299018338</li>\n",
       "\t<li>0.296428200455221</li>\n",
       "\t<li>0.658485137527882</li>\n",
       "\t<li>0.293742220481436</li>\n",
       "\t<li>0.615796688559526</li>\n",
       "</ol>\n"
      ],
      "text/latex": [
       "\\begin{enumerate*}\n",
       "\\item -0.840846299018338\n",
       "\\item 0.296428200455221\n",
       "\\item 0.658485137527882\n",
       "\\item 0.293742220481436\n",
       "\\item 0.615796688559526\n",
       "\\end{enumerate*}\n"
      ],
      "text/markdown": [
       "1. -0.840846299018338\n",
       "2. 0.296428200455221\n",
       "3. 0.658485137527882\n",
       "4. 0.293742220481436\n",
       "5. 0.615796688559526\n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "[1] -0.8408463  0.2964282  0.6584851  0.2937422  0.6157967"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "apply(mydata,2,mean, trim=0.2)  #计算每列的截尾均值，截尾均值基于中间60%的数据，最高和最低20%的值均被忽略"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "apply()是一种很强大的机制。apply()可把函数应用到数组的某个维度上，\n",
    "\n",
    "lapply()和sapply()则可将函数应用到列表（list）上。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5.3 数据处理难题的一套解决方案"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead><tr><th></th><th scope=col>Firstname</th><th scope=col>Lastname</th><th scope=col>Math</th><th scope=col>Science</th><th scope=col>English</th><th scope=col>score</th><th scope=col>grade</th></tr></thead>\n",
       "<tbody>\n",
       "\t<tr><th scope=row>6</th><td>Cheryl    </td><td>Cushing   </td><td>512       </td><td>85        </td><td>28        </td><td> 0.35     </td><td>C         </td></tr>\n",
       "\t<tr><th scope=row>1</th><td>John      </td><td>Davis     </td><td>502       </td><td>95        </td><td>25        </td><td> 0.56     </td><td>B         </td></tr>\n",
       "\t<tr><th scope=row>9</th><td>Joel      </td><td>England   </td><td>573       </td><td>89        </td><td>27        </td><td> 0.70     </td><td>B         </td></tr>\n",
       "\t<tr><th scope=row>4</th><td>David     </td><td>Jones     </td><td>358       </td><td>82        </td><td>15        </td><td>-1.16     </td><td>F         </td></tr>\n",
       "\t<tr><th scope=row>8</th><td>Greg      </td><td>Knox      </td><td>625       </td><td>95        </td><td>30        </td><td> 1.34     </td><td>A         </td></tr>\n",
       "\t<tr><th scope=row>5</th><td>Janice    </td><td>Markhammer</td><td>495       </td><td>75        </td><td>20        </td><td>-0.63     </td><td>D         </td></tr>\n",
       "\t<tr><th scope=row>3</th><td>Bullwinkle</td><td>Moose     </td><td>412       </td><td>80        </td><td>18        </td><td>-0.86     </td><td>D         </td></tr>\n",
       "\t<tr><th scope=row>10</th><td>Mary      </td><td>Rayburn   </td><td>522       </td><td>86        </td><td>18        </td><td>-0.18     </td><td>C         </td></tr>\n",
       "\t<tr><th scope=row>2</th><td>Angela    </td><td>Williams  </td><td>600       </td><td>99        </td><td>22        </td><td> 0.92     </td><td>A         </td></tr>\n",
       "\t<tr><th scope=row>7</th><td>Reuven    </td><td>Ytzrhak   </td><td>410       </td><td>80        </td><td>15        </td><td>-1.05     </td><td>F         </td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{r|lllllll}\n",
       "  & Firstname & Lastname & Math & Science & English & score & grade\\\\\n",
       "\\hline\n",
       "\t6 & Cheryl     & Cushing    & 512        & 85         & 28         &  0.35      & C         \\\\\n",
       "\t1 & John       & Davis      & 502        & 95         & 25         &  0.56      & B         \\\\\n",
       "\t9 & Joel       & England    & 573        & 89         & 27         &  0.70      & B         \\\\\n",
       "\t4 & David      & Jones      & 358        & 82         & 15         & -1.16      & F         \\\\\n",
       "\t8 & Greg       & Knox       & 625        & 95         & 30         &  1.34      & A         \\\\\n",
       "\t5 & Janice     & Markhammer & 495        & 75         & 20         & -0.63      & D         \\\\\n",
       "\t3 & Bullwinkle & Moose      & 412        & 80         & 18         & -0.86      & D         \\\\\n",
       "\t10 & Mary       & Rayburn    & 522        & 86         & 18         & -0.18      & C         \\\\\n",
       "\t2 & Angela     & Williams   & 600        & 99         & 22         &  0.92      & A         \\\\\n",
       "\t7 & Reuven     & Ytzrhak    & 410        & 80         & 15         & -1.05      & F         \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "| <!--/--> | Firstname | Lastname | Math | Science | English | score | grade | \n",
       "|---|---|---|---|---|---|---|---|---|---|\n",
       "| 6 | Cheryl     | Cushing    | 512        | 85         | 28         |  0.35      | C          | \n",
       "| 1 | John       | Davis      | 502        | 95         | 25         |  0.56      | B          | \n",
       "| 9 | Joel       | England    | 573        | 89         | 27         |  0.70      | B          | \n",
       "| 4 | David      | Jones      | 358        | 82         | 15         | -1.16      | F          | \n",
       "| 8 | Greg       | Knox       | 625        | 95         | 30         |  1.34      | A          | \n",
       "| 5 | Janice     | Markhammer | 495        | 75         | 20         | -0.63      | D          | \n",
       "| 3 | Bullwinkle | Moose      | 412        | 80         | 18         | -0.86      | D          | \n",
       "| 10 | Mary       | Rayburn    | 522        | 86         | 18         | -0.18      | C          | \n",
       "| 2 | Angela     | Williams   | 600        | 99         | 22         |  0.92      | A          | \n",
       "| 7 | Reuven     | Ytzrhak    | 410        | 80         | 15         | -1.05      | F          | \n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "   Firstname  Lastname   Math Science English score grade\n",
       "6  Cheryl     Cushing    512  85      28       0.35 C    \n",
       "1  John       Davis      502  95      25       0.56 B    \n",
       "9  Joel       England    573  89      27       0.70 B    \n",
       "4  David      Jones      358  82      15      -1.16 F    \n",
       "8  Greg       Knox       625  95      30       1.34 A    \n",
       "5  Janice     Markhammer 495  75      20      -0.63 D    \n",
       "3  Bullwinkle Moose      412  80      18      -0.86 D    \n",
       "10 Mary       Rayburn    522  86      18      -0.18 C    \n",
       "2  Angela     Williams   600  99      22       0.92 A    \n",
       "7  Reuven     Ytzrhak    410  80      15      -1.05 F    "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#将学生的各科考试成绩组合为单一的成绩衡量指标，基于相对名次（前20%、下20%、等等）给出从A到F的评分，根据学生姓氏和名字的首字母对花名册进行排序。\n",
    "\n",
    "options(digits=2)  #原始的学生花名册已经给出了。options(digits=2)限定了输出小数点后数字的位数，并且让输出更容易阅读\n",
    "Student <- c(\"John Davis\", \"Angela Williams\", \"Bullwinkle Moose\",\"David Jones\", \n",
    "             \"Janice Markhammer\", \"Cheryl Cushing\",\n",
    "             \"Reuven Ytzrhak\", \"Greg Knox\", \"Joel England\",\"Mary Rayburn\")\n",
    "Math <- c(502, 600, 412, 358, 495, 512, 410, 625, 573, 522)\n",
    "Science <- c(95, 99, 80, 82, 75, 85, 80, 95, 89, 86)\n",
    "English <- c(25, 22, 18, 15, 20, 28, 15, 30, 27, 18)\n",
    "roster <- data.frame(Student, Math, Science, English,\n",
    "                     stringsAsFactors=FALSE)\n",
    "\n",
    "z <- scale(roster[,2:4])\n",
    "#由于数学、科学和英语考试的分值不同（均值和标准差相去甚远），在组合之前需要先让它们变得可以比较。一种方法是将变量进行标准化，这样每科考试的成绩就都是用单位标准差来表示，而不是以原始的尺度来表示了。这个过程可以使用scale()函数来实现\n",
    "\n",
    "#计算综合得分\n",
    "score <- apply(z, 1, mean)\n",
    "roster <- cbind(roster, score)\n",
    "\n",
    "#函数quantile()给出了学生综合得分的百分位数\n",
    "y <- quantile(score, c(.8,.6,.4,.2))\n",
    "\n",
    "#通过使用逻辑运算符，你可以将学生的百分位数排名重编码为一个新的类别型成绩变量。\n",
    "roster$grade[score >= y[1]] <- \"A\"\n",
    "roster$grade[score < y[1] & score >= y[2]] <- \"B\"\n",
    "roster$grade[score < y[2] & score >= y[3]] <- \"C\"\n",
    "roster$grade[score < y[3] & score >= y[4]] <- \"D\"\n",
    "roster$grade[score < y[4]] <- \"F\"\n",
    "\n",
    "#使用函数strsplit()以空格为界把学生姓名拆分为姓氏和名字\n",
    "name <- strsplit((roster$Student), \" \")\n",
    "\n",
    "#使用函数sapply()提取列表中每个成分的第一个元素，放入一个储存名字的向量Firstname，并提取每个成分的第二个元素，放入一个储存姓氏的向量Lastname。\n",
    "Lastname <- sapply(name, \"[\", 2)\n",
    "Firstname <- sapply(name, \"[\", 1)\n",
    "roster <- cbind(Firstname,Lastname, roster[,-1])\n",
    "\n",
    "#可以使用函数order()依姓氏和名字对数据集进行排序\n",
    "roster <- roster[order(Lastname,Firstname),]\n",
    "roster"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5.4 控制流"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.4.1 重复和循环"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. for结构"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "#for循环重复地执行一个语句，直到某个变量的值不再包含在序列seq中为止。\n",
    "for (var in seq) statement"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1] \"Hello\"\n",
      "[1] \"Hello\"\n",
      "[1] \"Hello\"\n",
      "[1] \"Hello\"\n",
      "[1] \"Hello\"\n",
      "[1] \"Hello\"\n",
      "[1] \"Hello\"\n",
      "[1] \"Hello\"\n",
      "[1] \"Hello\"\n",
      "[1] \"Hello\"\n"
     ]
    }
   ],
   "source": [
    "for (i in 1:10) print(\"Hello\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. while结构"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#while循环重复地执行一个语句，直到条件不为真为止。语法为：\n",
    "\n",
    "while (cond) statement"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1] \"Hello\"\n",
      "[1] \"Hello\"\n",
      "[1] \"Hello\"\n",
      "[1] \"Hello\"\n",
      "[1] \"Hello\"\n",
      "[1] \"Hello\"\n",
      "[1] \"Hello\"\n",
      "[1] \"Hello\"\n",
      "[1] \"Hello\"\n",
      "[1] \"Hello\"\n"
     ]
    }
   ],
   "source": [
    "i <- 10\n",
    "while (i > 0) {\n",
    "    print(\"Hello\"); \n",
    "    i <- i - 1}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.4.2 条件执行"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. if-else结构"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "控制结构if-else在某个给定条件为真时执行语句。也可以同时在条件为假时执行另外的语句。语法为：\n",
    "    \n",
    "if (cond) statement\n",
    "\n",
    "if (cond) statement1 else statement2"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "if (is.character(grade)) grade <- as.factor(grade)\n",
    "\n",
    "if (!is.factor(grade)) grade <- as.factor(grade) else print(\"Grade already is a factor\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. ifelse结构"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ifelse结构是if-else结构比较紧凑的向量化版本，其语法为：\n",
    "    \n",
    "ifelse(cond, statement1, statement2)\n",
    "\n",
    "若cond为TRUE，则执行第一个语句；若cond为FALSE，则执行第二个语句。"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "ifelse(score > 0.5, print(\"Passed\"), print(\"Failed\"))\n",
    "\n",
    "outcome <- ifelse (score > 0.5, \"Passed\", \"Failed\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. switch结构"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "switch根据一个表达式的值选择语句执行。语法为：\n",
    "\n",
    "switch(expr, ...)\n",
    "\n",
    "其中的...表示与expr的各种可能输出值绑定的语句."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1] \"Cheer up\"\n",
      "[1] \"There is nothing to fear\"\n"
     ]
    }
   ],
   "source": [
    "#一个switch示例\n",
    "\n",
    "feelings <- c(\"sad\", \"afraid\")\n",
    "for (i in feelings)\n",
    "    print(\n",
    "        switch(i,\n",
    "               happy = \"I am glad you are happy\",\n",
    "               afraid = \"There is nothing to fear\",\n",
    "               sad = \"Cheer up\",\n",
    "               angry = \"Calm down now\"\n",
    "              )\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.5 用户自编函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "一个函数的结构看起来大致如此："
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "myfunction <- function(arg1, arg2, ... ){\n",
    "    statements\n",
    "    return(object)\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "#mystats()：一个由用户编写的描述性统计量计算函数\n",
    "\n",
    "mystats <- function(x, parametric=TRUE, print=FALSE) {\n",
    "    if (parametric) {\n",
    "        center <- mean(x); spread <- sd(x)\n",
    "    } else {\n",
    "        center <- median(x); spread <- mad(x)\n",
    "    }\n",
    "    if (print & parametric) {\n",
    "        cat(\"Mean=\", center, \"\\n\", \"SD=\", spread, \"\\n\")\n",
    "    } else if (\n",
    "        print & !parametric) {\n",
    "        cat(\"Median=\", center, \"\\n\", \"MAD=\", spread, \"\\n\")\n",
    "    }\n",
    "    result <- list(center=center, spread=spread)\n",
    "    return(result)\n",
    "}\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<dl>\n",
       "\t<dt>$center</dt>\n",
       "\t\t<dd>0.00183882061433999</dd>\n",
       "\t<dt>$spread</dt>\n",
       "\t\t<dd>1.03481394576807</dd>\n",
       "</dl>\n"
      ],
      "text/latex": [
       "\\begin{description}\n",
       "\\item[\\$center] 0.00183882061433999\n",
       "\\item[\\$spread] 1.03481394576807\n",
       "\\end{description}\n"
      ],
      "text/markdown": [
       "$center\n",
       ":   0.00183882061433999\n",
       "$spread\n",
       ":   1.03481394576807\n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "$center\n",
       "[1] 0.0018\n",
       "\n",
       "$spread\n",
       "[1] 1\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "set.seed(1234)\n",
    "x <- rnorm(500)\n",
    "y <- mystats(x)\n",
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 5.6 整合与重构"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "R中提供了许多用来整合（aggregate）和重塑（reshape）数据的强大方法。在整合数据时，往往将多组观测替换为根据这些观测计算的描述性统计量。在重塑数据时，则会通过修改数据的结构（行和列）来决定数据的组织方式。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.6.1 转置"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "转置（反转行和列）也许是重塑数据集的众多方法中最简单的一个了。使用函数t()即可对一个矩阵或数据框进行转置。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead><tr><th></th><th scope=col>mpg</th><th scope=col>cyl</th><th scope=col>disp</th><th scope=col>hp</th></tr></thead>\n",
       "<tbody>\n",
       "\t<tr><th scope=row>Mazda RX4</th><td>21.0</td><td>6   </td><td>160 </td><td>110 </td></tr>\n",
       "\t<tr><th scope=row>Mazda RX4 Wag</th><td>21.0</td><td>6   </td><td>160 </td><td>110 </td></tr>\n",
       "\t<tr><th scope=row>Datsun 710</th><td>22.8</td><td>4   </td><td>108 </td><td> 93 </td></tr>\n",
       "\t<tr><th scope=row>Hornet 4 Drive</th><td>21.4</td><td>6   </td><td>258 </td><td>110 </td></tr>\n",
       "\t<tr><th scope=row>Hornet Sportabout</th><td>18.7</td><td>8   </td><td>360 </td><td>175 </td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{r|llll}\n",
       "  & mpg & cyl & disp & hp\\\\\n",
       "\\hline\n",
       "\tMazda RX4 & 21.0 & 6    & 160  & 110 \\\\\n",
       "\tMazda RX4 Wag & 21.0 & 6    & 160  & 110 \\\\\n",
       "\tDatsun 710 & 22.8 & 4    & 108  &  93 \\\\\n",
       "\tHornet 4 Drive & 21.4 & 6    & 258  & 110 \\\\\n",
       "\tHornet Sportabout & 18.7 & 8    & 360  & 175 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "| <!--/--> | mpg | cyl | disp | hp | \n",
       "|---|---|---|---|---|\n",
       "| Mazda RX4 | 21.0 | 6    | 160  | 110  | \n",
       "| Mazda RX4 Wag | 21.0 | 6    | 160  | 110  | \n",
       "| Datsun 710 | 22.8 | 4    | 108  |  93  | \n",
       "| Hornet 4 Drive | 21.4 | 6    | 258  | 110  | \n",
       "| Hornet Sportabout | 18.7 | 8    | 360  | 175  | \n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "                  mpg  cyl disp hp \n",
       "Mazda RX4         21.0 6   160  110\n",
       "Mazda RX4 Wag     21.0 6   160  110\n",
       "Datsun 710        22.8 4   108   93\n",
       "Hornet 4 Drive    21.4 6   258  110\n",
       "Hornet Sportabout 18.7 8   360  175"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead><tr><th></th><th scope=col>Mazda RX4</th><th scope=col>Mazda RX4 Wag</th><th scope=col>Datsun 710</th><th scope=col>Hornet 4 Drive</th><th scope=col>Hornet Sportabout</th></tr></thead>\n",
       "<tbody>\n",
       "\t<tr><th scope=row>mpg</th><td> 21  </td><td> 21  </td><td> 22.8</td><td> 21.4</td><td> 18.7</td></tr>\n",
       "\t<tr><th scope=row>cyl</th><td>  6  </td><td>  6  </td><td>  4.0</td><td>  6.0</td><td>  8.0</td></tr>\n",
       "\t<tr><th scope=row>disp</th><td>160  </td><td>160  </td><td>108.0</td><td>258.0</td><td>360.0</td></tr>\n",
       "\t<tr><th scope=row>hp</th><td>110  </td><td>110  </td><td> 93.0</td><td>110.0</td><td>175.0</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{r|lllll}\n",
       "  & Mazda RX4 & Mazda RX4 Wag & Datsun 710 & Hornet 4 Drive & Hornet Sportabout\\\\\n",
       "\\hline\n",
       "\tmpg &  21   &  21   &  22.8 &  21.4 &  18.7\\\\\n",
       "\tcyl &   6   &   6   &   4.0 &   6.0 &   8.0\\\\\n",
       "\tdisp & 160   & 160   & 108.0 & 258.0 & 360.0\\\\\n",
       "\thp & 110   & 110   &  93.0 & 110.0 & 175.0\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "| <!--/--> | Mazda RX4 | Mazda RX4 Wag | Datsun 710 | Hornet 4 Drive | Hornet Sportabout | \n",
       "|---|---|---|---|\n",
       "| mpg |  21   |  21   |  22.8 |  21.4 |  18.7 | \n",
       "| cyl |   6   |   6   |   4.0 |   6.0 |   8.0 | \n",
       "| disp | 160   | 160   | 108.0 | 258.0 | 360.0 | \n",
       "| hp | 110   | 110   |  93.0 | 110.0 | 175.0 | \n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "     Mazda RX4 Mazda RX4 Wag Datsun 710 Hornet 4 Drive Hornet Sportabout\n",
       "mpg   21        21            22.8       21.4           18.7            \n",
       "cyl    6         6             4.0        6.0            8.0            \n",
       "disp 160       160           108.0      258.0          360.0            \n",
       "hp   110       110            93.0      110.0          175.0            "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#数据集的转置\n",
    "cars <- mtcars[1:5,1:4]\n",
    "cars\n",
    "t(cars)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.6.2 整合数据"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在R中使用一个或多个by变量和一个预先定义好的函数来折叠（collapse）数据是比较容易的。调用格式为：\n",
    "\n",
    "aggregate(x, by, FUN)\n",
    "\n",
    "x是待折叠的数据对象，\n",
    "\n",
    "by是一个变量名组成的列表，这些变量将被去掉以形成新的观测，\n",
    "\n",
    "FUN则是用来计算描述性统计量的标量函数，它将被用来计算新观测中的值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "The following objects are masked from mtcars (pos = 3):\n",
      "\n",
      "    am, carb, cyl, disp, drat, gear, hp, mpg, qsec, vs, wt\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table>\n",
       "<thead><tr><th scope=col>Group.1</th><th scope=col>Group.2</th><th scope=col>mpg</th><th scope=col>cyl</th><th scope=col>disp</th><th scope=col>hp</th><th scope=col>drat</th><th scope=col>wt</th><th scope=col>qsec</th><th scope=col>vs</th><th scope=col>am</th><th scope=col>gear</th><th scope=col>carb</th></tr></thead>\n",
       "<tbody>\n",
       "\t<tr><td>4   </td><td>3   </td><td>21.5</td><td>4   </td><td>120 </td><td> 97 </td><td>3.70</td><td>2.46</td><td>20.0</td><td>1.0 </td><td>0.00</td><td>3   </td><td>1.00</td></tr>\n",
       "\t<tr><td>6   </td><td>3   </td><td>19.8</td><td>6   </td><td>242 </td><td>108 </td><td>2.92</td><td>3.34</td><td>19.8</td><td>1.0 </td><td>0.00</td><td>3   </td><td>1.00</td></tr>\n",
       "\t<tr><td>8   </td><td>3   </td><td>15.1</td><td>8   </td><td>358 </td><td>194 </td><td>3.12</td><td>4.10</td><td>17.1</td><td>0.0 </td><td>0.00</td><td>3   </td><td>3.08</td></tr>\n",
       "\t<tr><td>4   </td><td>4   </td><td>26.9</td><td>4   </td><td>103 </td><td> 76 </td><td>4.11</td><td>2.38</td><td>19.6</td><td>1.0 </td><td>0.75</td><td>4   </td><td>1.50</td></tr>\n",
       "\t<tr><td>6   </td><td>4   </td><td>19.8</td><td>6   </td><td>164 </td><td>116 </td><td>3.91</td><td>3.09</td><td>17.7</td><td>0.5 </td><td>0.50</td><td>4   </td><td>4.00</td></tr>\n",
       "\t<tr><td>4   </td><td>5   </td><td>28.2</td><td>4   </td><td>108 </td><td>102 </td><td>4.10</td><td>1.83</td><td>16.8</td><td>0.5 </td><td>1.00</td><td>5   </td><td>2.00</td></tr>\n",
       "\t<tr><td>6   </td><td>5   </td><td>19.7</td><td>6   </td><td>145 </td><td>175 </td><td>3.62</td><td>2.77</td><td>15.5</td><td>0.0 </td><td>1.00</td><td>5   </td><td>6.00</td></tr>\n",
       "\t<tr><td>8   </td><td>5   </td><td>15.4</td><td>8   </td><td>326 </td><td>300 </td><td>3.88</td><td>3.37</td><td>14.6</td><td>0.0 </td><td>1.00</td><td>5   </td><td>6.00</td></tr>\n",
       "</tbody>\n",
       "</table>\n"
      ],
      "text/latex": [
       "\\begin{tabular}{r|lllllllllllll}\n",
       " Group.1 & Group.2 & mpg & cyl & disp & hp & drat & wt & qsec & vs & am & gear & carb\\\\\n",
       "\\hline\n",
       "\t 4    & 3    & 21.5 & 4    & 120  &  97  & 3.70 & 2.46 & 20.0 & 1.0  & 0.00 & 3    & 1.00\\\\\n",
       "\t 6    & 3    & 19.8 & 6    & 242  & 108  & 2.92 & 3.34 & 19.8 & 1.0  & 0.00 & 3    & 1.00\\\\\n",
       "\t 8    & 3    & 15.1 & 8    & 358  & 194  & 3.12 & 4.10 & 17.1 & 0.0  & 0.00 & 3    & 3.08\\\\\n",
       "\t 4    & 4    & 26.9 & 4    & 103  &  76  & 4.11 & 2.38 & 19.6 & 1.0  & 0.75 & 4    & 1.50\\\\\n",
       "\t 6    & 4    & 19.8 & 6    & 164  & 116  & 3.91 & 3.09 & 17.7 & 0.5  & 0.50 & 4    & 4.00\\\\\n",
       "\t 4    & 5    & 28.2 & 4    & 108  & 102  & 4.10 & 1.83 & 16.8 & 0.5  & 1.00 & 5    & 2.00\\\\\n",
       "\t 6    & 5    & 19.7 & 6    & 145  & 175  & 3.62 & 2.77 & 15.5 & 0.0  & 1.00 & 5    & 6.00\\\\\n",
       "\t 8    & 5    & 15.4 & 8    & 326  & 300  & 3.88 & 3.37 & 14.6 & 0.0  & 1.00 & 5    & 6.00\\\\\n",
       "\\end{tabular}\n"
      ],
      "text/markdown": [
       "\n",
       "Group.1 | Group.2 | mpg | cyl | disp | hp | drat | wt | qsec | vs | am | gear | carb | \n",
       "|---|---|---|---|---|---|---|---|\n",
       "| 4    | 3    | 21.5 | 4    | 120  |  97  | 3.70 | 2.46 | 20.0 | 1.0  | 0.00 | 3    | 1.00 | \n",
       "| 6    | 3    | 19.8 | 6    | 242  | 108  | 2.92 | 3.34 | 19.8 | 1.0  | 0.00 | 3    | 1.00 | \n",
       "| 8    | 3    | 15.1 | 8    | 358  | 194  | 3.12 | 4.10 | 17.1 | 0.0  | 0.00 | 3    | 3.08 | \n",
       "| 4    | 4    | 26.9 | 4    | 103  |  76  | 4.11 | 2.38 | 19.6 | 1.0  | 0.75 | 4    | 1.50 | \n",
       "| 6    | 4    | 19.8 | 6    | 164  | 116  | 3.91 | 3.09 | 17.7 | 0.5  | 0.50 | 4    | 4.00 | \n",
       "| 4    | 5    | 28.2 | 4    | 108  | 102  | 4.10 | 1.83 | 16.8 | 0.5  | 1.00 | 5    | 2.00 | \n",
       "| 6    | 5    | 19.7 | 6    | 145  | 175  | 3.62 | 2.77 | 15.5 | 0.0  | 1.00 | 5    | 6.00 | \n",
       "| 8    | 5    | 15.4 | 8    | 326  | 300  | 3.88 | 3.37 | 14.6 | 0.0  | 1.00 | 5    | 6.00 | \n",
       "\n",
       "\n"
      ],
      "text/plain": [
       "  Group.1 Group.2 mpg  cyl disp hp  drat wt   qsec vs  am   gear carb\n",
       "1 4       3       21.5 4   120   97 3.70 2.46 20.0 1.0 0.00 3    1.00\n",
       "2 6       3       19.8 6   242  108 2.92 3.34 19.8 1.0 0.00 3    1.00\n",
       "3 8       3       15.1 8   358  194 3.12 4.10 17.1 0.0 0.00 3    3.08\n",
       "4 4       4       26.9 4   103   76 4.11 2.38 19.6 1.0 0.75 4    1.50\n",
       "5 6       4       19.8 6   164  116 3.91 3.09 17.7 0.5 0.50 4    4.00\n",
       "6 4       5       28.2 4   108  102 4.10 1.83 16.8 0.5 1.00 5    2.00\n",
       "7 6       5       19.7 6   145  175 3.62 2.77 15.5 0.0 1.00 5    6.00\n",
       "8 8       5       15.4 8   326  300 3.88 3.37 14.6 0.0 1.00 5    6.00"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#整合数据\n",
    "options(digits=3)\n",
    "attach(mtcars)\n",
    "aggdata <-aggregate(mtcars, by=list(cyl,gear), FUN=mean, na.rm=TRUE)\n",
    "aggdata"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在结果中，Group.1表示汽缸数量（4、6或8），Group.2代表挡位数（3、4或5）。举例来说，\n",
    "拥有4个汽缸和3个挡位车型的每加仑汽油行驶英里数（mpg）均值为21.5。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.6.3 reshape2 包"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "reshape2包是一套重构和整合数据集的绝妙的万能工具。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "大致说来，你需要首先将数据融合（melt），以使每一行都是唯一的标识符变量组合。然后\n",
    "将数据重铸（cast）为你想要的任何形状。在重铸过程中，你可以使用任何函数对数据进行整合。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. 融合"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "数据集的融合是将它重构为这样一种格式：每个测量变量独占一行，行中带有要唯一确定这\n",
    "个测量所需的标识符变量。"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "library(reshape2)\n",
    "md <- melt(mydata, id=c(\"ID\", \"Time\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. 重铸"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "dcast()函数读取已融合的数据，并使用你提供的公式和一个（可选的）用于整合数据的函\n",
    "数将其重塑。调用格式为：\n",
    "\n",
    "newdata <- dcast(md, formula, fun.aggregate)\n",
    "\n",
    "其中的md为已融合的数据，formula描述了想要的最后结果，而fun.aggregate是（可选\n",
    "的）数据整合函数。其接受的公式形如：\n",
    "\n",
    "rowvar1 + rowvar2 + ... ~ colvar1 + colvar2 + ...\n",
    "\n",
    "在这一公式中，rowvar1 + rowvar2 + ...定义了要划掉的变量集合，以确定各行的内容，\n",
    "而colvar1 + colvar2 + ...则定义了要划掉的、确定各列内容的变量集合。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "R",
   "language": "R",
   "name": "ir"
  },
  "language_info": {
   "codemirror_mode": "r",
   "file_extension": ".r",
   "mimetype": "text/x-r-source",
   "name": "R",
   "pygments_lexer": "r",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}