parallel task di python.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Parallel task di python\n",
    "\n",
    "Untuk menjalankan program secara parallel atau async, python menyediakan module `concurrent.futures` (python versi 3.2 keatas). Terdapat `ThreadPoolExecutor` untuk menjalankan di threads berbeda, dan `ProcessPoolExecutor` untuk menjalankan di proses yang berbeda.\n",
    "\n",
    "\n",
    "`ThreadPoolExecutor` bagus digunakan untuk task yang prosesnya berjalan di `io` seperti network access, disk, dll\n",
    "\n",
    "`ProcessPoolExecutor` bagus digunakan untuk task dengan proses yang berjalan di cpu. Seperti image resize, encoding, dll"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ThreadPoolExecutor\n",
    "\n",
    "Contoh memanggil rest api secara bersamaan"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "start request\n",
      "end request\n",
      "start request\n",
      "end request\n",
      "{'hello': 'world'}\n",
      "{'hello': 'world'}\n",
      "executed in: 5.20\n"
     ]
    }
   ],
   "source": [
    "import requests\n",
    "import time\n",
    "\n",
    "def call_api():\n",
    "    print('start request')\n",
    "    r = requests.get(f'http://www.mocky.io/v2/5d014d8d3200002800f9db80?mocky-delay=2s')\n",
    "    print('end request')\n",
    "    return r.json()\n",
    "\n",
    "# normal call\n",
    "start = time.time()\n",
    "task1 = call_api()\n",
    "task2 = call_api()\n",
    "\n",
    "print(task1)\n",
    "print(task2)\n",
    "print('executed in: %.2f' % (time.time() - start))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "start requeststart request\n",
      "\n",
      "end request\n",
      "{'hello': 'world'}\n",
      "end request\n",
      "{'hello': 'world'}\n",
      "executed in: 2.55\n"
     ]
    }
   ],
   "source": [
    "from concurrent.futures import ThreadPoolExecutor\n",
    "\n",
    "# using threadpool\n",
    "start = time.time()\n",
    "with ThreadPoolExecutor() as executor:\n",
    "    task1 = executor.submit(call_api)\n",
    "    task2 = executor.submit(call_api)\n",
    "    print(task1.result())\n",
    "    print(task2.result())\n",
    "print('executed in: %.2f' % (time.time() - start))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Bisa dilihat bahwa `call_api` dipanggil bersamaan saat menggunakan `ThreadPoolExecutor`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Inside class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "from concurrent.futures import ProcessPoolExecutor\n",
    "\n",
    "\n",
    "class MyClass:\n",
    "    \n",
    "    def job(self):\n",
    "        print('start request')\n",
    "        r = requests.get(f'http://www.mocky.io/v2/5d014d8d3200002800f9db80?mocky-delay=2s')\n",
    "        print('end request')\n",
    "        return r.json()\n",
    "    \n",
    "    def run(self):\n",
    "        start = time.time()\n",
    "        with ProcessPoolExecutor() as executor:\n",
    "            task1 = executor.submit(self.job)\n",
    "            task2 = executor.submit(self.job)\n",
    "            print(task1.result())\n",
    "            print(task2.result())\n",
    "        print('executed in: %.2f' % (time.time() - start))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "start request\n",
      "start request\n",
      "end request\n",
      "end request\n",
      "{'hello': 'world'}\n",
      "{'hello': 'world'}\n",
      "executed in: 2.59\n"
     ]
    }
   ],
   "source": [
    "a = MyClass()\n",
    "a.run()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}