data-wrangler.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data wrangling of the IEA's *Net Zero by 2050 Roadmap* (2021)\n",
    "\n",
    "This notebook transform the data supporting the\n",
    "IEA's [Net Zero by 2050](https://www.iea.org/reports/net-zero-by-2050) flagship report\n",
    "(published May 2021)  \n",
    "to the IAMC format using the **pyam** package ([read the docs](https://pyam-iamc.readthedocs.io))."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/javascript": [
       "IPython.OutputArea.prototype._should_scroll = function(lines) { return false; }"
      ],
      "text/plain": [
       "<IPython.core.display.Javascript object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import pyam"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "index = dict(model=\"IEA\", scenario=\"Netzero2050\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Annex A\n",
    "\n",
    "Annex A has the scenario projections supporting and quantifying the \"Net Zero by 2050 Roadmap\".\n",
    "\n",
    "This script uses the files as published on the IEA data page on May 18, 2021.  \n",
    "Unfortunately, the IEA does not implement clear versioning of the files, so this script may not work as is if the IEA decides to update these files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "file = \"data/NZE2021_AnnexA.xlsx\"\n",
    "df_list = []  # this list will hold the individual refactored IamDataFrames"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Global energy supply (\"World_Balance\")\n",
    "\n",
    "#### Total Energy Supply\n",
    "\n",
    "Read the relevant section of the World_Balance sheet and cast to an **IamDataFrame**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "balances_totals = pd.read_excel(file, sheet_name=\"World_Balance\", header=4, usecols=\"A:F\")[0:17]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_balances_totals = pyam.IamDataFrame(balances_totals, **index, region=\"World\", variable=\"Unnamed: 0\", unit=\"EJ\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Rename the fuels to IAMC-style variables similar to those used in the IPCC AR6."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "variable_mapping = {\n",
    "    \"Total energy supply\" : \"Primary Energy\",\n",
    "    \"Modern gaseous bioenergy\": \"Bioenergy|Gaseous\",\n",
    "    \"Modern liquid bioenergy\": \"Bioenergy|Liquid\",\n",
    "    \"Modern solid bioenergy\": \"Bioenergy|Solid\",\n",
    "    \"Renewables\": \"Renewables (excluding traditional biomass)\",\n",
    "    \"Traditional use of biomass\": \"Biomass|Traditional\",\n",
    "    \"Coal with CCUS\": \"Coal|CCUS\",\n",
    "    \"Unabated coal\": \"Coal|Unabated\",\n",
    "    \"Natural gas with CCUS\": \"Natural Gas|CCUS\" ,\n",
    "    \"Unabated natural gas\": \"Natural Gas|Unabated\",\n",
    "    \"of which non-energy use\": \"Oil|Non-energy Use\"\n",
    "}\n",
    "\n",
    "df_balances_totals.rename(variable=variable_mapping, inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prefix \"Primary Energy\" to variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "prefix = \"Primary Energy\"\n",
    "\n",
    "df_balances_totals.rename(\n",
    "    variable=dict([(v, f\"{prefix}|{v}\") for v in df_balances_totals.variable if v != prefix]),\n",
    "    inplace=True\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compute aggregates of timeseries data that are only given at sub-sectoral detail"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "agg_vars = [\"Primary Energy|Natural Gas\", \"Primary Energy|Coal\", \"Primary Energy|Bioenergy\"]\n",
    "\n",
    "for v in agg_vars:\n",
    "    df_balances_totals.aggregate(v, append=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Show the timeseries data in the IAMC format."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>2019</th>\n",
       "      <th>2020</th>\n",
       "      <th>2030</th>\n",
       "      <th>2040</th>\n",
       "      <th>2050</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>model</th>\n",
       "      <th>scenario</th>\n",
       "      <th>region</th>\n",
       "      <th>variable</th>\n",
       "      <th>unit</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"20\" valign=\"top\">IEA</th>\n",
       "      <th rowspan=\"20\" valign=\"top\">Netzero2050</th>\n",
       "      <th rowspan=\"20\" valign=\"top\">World</th>\n",
       "      <th>Primary Energy</th>\n",
       "      <th>EJ</th>\n",
       "      <td>612.0</td>\n",
       "      <td>587.0</td>\n",
       "      <td>547.0</td>\n",
       "      <td>535.0</td>\n",
       "      <td>543.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Primary Energy|Bioenergy</th>\n",
       "      <th>EJ</th>\n",
       "      <td>37.0</td>\n",
       "      <td>37.0</td>\n",
       "      <td>71.0</td>\n",
       "      <td>97.0</td>\n",
       "      <td>102.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Primary Energy|Bioenergy|Gaseous</th>\n",
       "      <th>EJ</th>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>14.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Primary Energy|Bioenergy|Liquid</th>\n",
       "      <th>EJ</th>\n",
       "      <td>4.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>12.0</td>\n",
       "      <td>14.0</td>\n",
       "      <td>15.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Primary Energy|Bioenergy|Solid</th>\n",
       "      <th>EJ</th>\n",
       "      <td>31.0</td>\n",
       "      <td>32.0</td>\n",
       "      <td>54.0</td>\n",
       "      <td>73.0</td>\n",
       "      <td>73.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Primary Energy|Biomass|Traditional</th>\n",
       "      <th>EJ</th>\n",
       "      <td>25.0</td>\n",
       "      <td>25.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Primary Energy|Coal</th>\n",
       "      <th>EJ</th>\n",
       "      <td>160.0</td>\n",
       "      <td>154.0</td>\n",
       "      <td>72.0</td>\n",
       "      <td>32.0</td>\n",
       "      <td>17.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Primary Energy|Coal|CCUS</th>\n",
       "      <th>EJ</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>16.0</td>\n",
       "      <td>14.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Primary Energy|Coal|Unabated</th>\n",
       "      <th>EJ</th>\n",
       "      <td>160.0</td>\n",
       "      <td>154.0</td>\n",
       "      <td>68.0</td>\n",
       "      <td>16.0</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Primary Energy|Hydro</th>\n",
       "      <th>EJ</th>\n",
       "      <td>15.0</td>\n",
       "      <td>16.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>27.0</td>\n",
       "      <td>30.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Primary Energy|Natural Gas</th>\n",
       "      <th>EJ</th>\n",
       "      <td>139.0</td>\n",
       "      <td>137.0</td>\n",
       "      <td>129.0</td>\n",
       "      <td>75.0</td>\n",
       "      <td>60.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Primary Energy|Natural Gas|CCUS</th>\n",
       "      <th>EJ</th>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>31.0</td>\n",
       "      <td>43.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Primary Energy|Natural Gas|Unabated</th>\n",
       "      <th>EJ</th>\n",
       "      <td>139.0</td>\n",
       "      <td>136.0</td>\n",
       "      <td>116.0</td>\n",
       "      <td>44.0</td>\n",
       "      <td>17.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Primary Energy|Nuclear</th>\n",
       "      <th>EJ</th>\n",
       "      <td>30.0</td>\n",
       "      <td>29.0</td>\n",
       "      <td>41.0</td>\n",
       "      <td>54.0</td>\n",
       "      <td>61.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Primary Energy|Oil</th>\n",
       "      <th>EJ</th>\n",
       "      <td>190.0</td>\n",
       "      <td>173.0</td>\n",
       "      <td>137.0</td>\n",
       "      <td>79.0</td>\n",
       "      <td>42.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Primary Energy|Oil|Non-energy Use</th>\n",
       "      <th>EJ</th>\n",
       "      <td>28.0</td>\n",
       "      <td>27.0</td>\n",
       "      <td>32.0</td>\n",
       "      <td>31.0</td>\n",
       "      <td>29.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Primary Energy|Other renewables</th>\n",
       "      <th>EJ</th>\n",
       "      <td>4.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>13.0</td>\n",
       "      <td>24.0</td>\n",
       "      <td>32.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Primary Energy|Renewables (excluding traditional biomass)</th>\n",
       "      <th>EJ</th>\n",
       "      <td>67.0</td>\n",
       "      <td>69.0</td>\n",
       "      <td>167.0</td>\n",
       "      <td>295.0</td>\n",
       "      <td>362.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Primary Energy|Solar</th>\n",
       "      <th>EJ</th>\n",
       "      <td>4.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>32.0</td>\n",
       "      <td>78.0</td>\n",
       "      <td>109.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Primary Energy|Wind</th>\n",
       "      <th>EJ</th>\n",
       "      <td>5.0</td>\n",
       "      <td>6.0</td>\n",
       "      <td>29.0</td>\n",
       "      <td>67.0</td>\n",
       "      <td>89.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                                   2019  \\\n",
       "model scenario    region variable                                           unit          \n",
       "IEA   Netzero2050 World  Primary Energy                                     EJ    612.0   \n",
       "                         Primary Energy|Bioenergy                           EJ     37.0   \n",
       "                         Primary Energy|Bioenergy|Gaseous                   EJ      2.0   \n",
       "                         Primary Energy|Bioenergy|Liquid                    EJ      4.0   \n",
       "                         Primary Energy|Bioenergy|Solid                     EJ     31.0   \n",
       "                         Primary Energy|Biomass|Traditional                 EJ     25.0   \n",
       "                         Primary Energy|Coal                                EJ    160.0   \n",
       "                         Primary Energy|Coal|CCUS                           EJ      0.0   \n",
       "                         Primary Energy|Coal|Unabated                       EJ    160.0   \n",
       "                         Primary Energy|Hydro                               EJ     15.0   \n",
       "                         Primary Energy|Natural Gas                         EJ    139.0   \n",
       "                         Primary Energy|Natural Gas|CCUS                    EJ      0.0   \n",
       "                         Primary Energy|Natural Gas|Unabated                EJ    139.0   \n",
       "                         Primary Energy|Nuclear                             EJ     30.0   \n",
       "                         Primary Energy|Oil                                 EJ    190.0   \n",
       "                         Primary Energy|Oil|Non-energy Use                  EJ     28.0   \n",
       "                         Primary Energy|Other renewables                    EJ      4.0   \n",
       "                         Primary Energy|Renewables (excluding traditiona... EJ     67.0   \n",
       "                         Primary Energy|Solar                               EJ      4.0   \n",
       "                         Primary Energy|Wind                                EJ      5.0   \n",
       "\n",
       "                                                                                   2020  \\\n",
       "model scenario    region variable                                           unit          \n",
       "IEA   Netzero2050 World  Primary Energy                                     EJ    587.0   \n",
       "                         Primary Energy|Bioenergy                           EJ     37.0   \n",
       "                         Primary Energy|Bioenergy|Gaseous                   EJ      2.0   \n",
       "                         Primary Energy|Bioenergy|Liquid                    EJ      3.0   \n",
       "                         Primary Energy|Bioenergy|Solid                     EJ     32.0   \n",
       "                         Primary Energy|Biomass|Traditional                 EJ     25.0   \n",
       "                         Primary Energy|Coal                                EJ    154.0   \n",
       "                         Primary Energy|Coal|CCUS                           EJ      0.0   \n",
       "                         Primary Energy|Coal|Unabated                       EJ    154.0   \n",
       "                         Primary Energy|Hydro                               EJ     16.0   \n",
       "                         Primary Energy|Natural Gas                         EJ    137.0   \n",
       "                         Primary Energy|Natural Gas|CCUS                    EJ      1.0   \n",
       "                         Primary Energy|Natural Gas|Unabated                EJ    136.0   \n",
       "                         Primary Energy|Nuclear                             EJ     29.0   \n",
       "                         Primary Energy|Oil                                 EJ    173.0   \n",
       "                         Primary Energy|Oil|Non-energy Use                  EJ     27.0   \n",
       "                         Primary Energy|Other renewables                    EJ      5.0   \n",
       "                         Primary Energy|Renewables (excluding traditiona... EJ     69.0   \n",
       "                         Primary Energy|Solar                               EJ      5.0   \n",
       "                         Primary Energy|Wind                                EJ      6.0   \n",
       "\n",
       "                                                                                   2030  \\\n",
       "model scenario    region variable                                           unit          \n",
       "IEA   Netzero2050 World  Primary Energy                                     EJ    547.0   \n",
       "                         Primary Energy|Bioenergy                           EJ     71.0   \n",
       "                         Primary Energy|Bioenergy|Gaseous                   EJ      5.0   \n",
       "                         Primary Energy|Bioenergy|Liquid                    EJ     12.0   \n",
       "                         Primary Energy|Bioenergy|Solid                     EJ     54.0   \n",
       "                         Primary Energy|Biomass|Traditional                 EJ      0.0   \n",
       "                         Primary Energy|Coal                                EJ     72.0   \n",
       "                         Primary Energy|Coal|CCUS                           EJ      4.0   \n",
       "                         Primary Energy|Coal|Unabated                       EJ     68.0   \n",
       "                         Primary Energy|Hydro                               EJ     21.0   \n",
       "                         Primary Energy|Natural Gas                         EJ    129.0   \n",
       "                         Primary Energy|Natural Gas|CCUS                    EJ     13.0   \n",
       "                         Primary Energy|Natural Gas|Unabated                EJ    116.0   \n",
       "                         Primary Energy|Nuclear                             EJ     41.0   \n",
       "                         Primary Energy|Oil                                 EJ    137.0   \n",
       "                         Primary Energy|Oil|Non-energy Use                  EJ     32.0   \n",
       "                         Primary Energy|Other renewables                    EJ     13.0   \n",
       "                         Primary Energy|Renewables (excluding traditiona... EJ    167.0   \n",
       "                         Primary Energy|Solar                               EJ     32.0   \n",
       "                         Primary Energy|Wind                                EJ     29.0   \n",
       "\n",
       "                                                                                   2040  \\\n",
       "model scenario    region variable                                           unit          \n",
       "IEA   Netzero2050 World  Primary Energy                                     EJ    535.0   \n",
       "                         Primary Energy|Bioenergy                           EJ     97.0   \n",
       "                         Primary Energy|Bioenergy|Gaseous                   EJ     10.0   \n",
       "                         Primary Energy|Bioenergy|Liquid                    EJ     14.0   \n",
       "                         Primary Energy|Bioenergy|Solid                     EJ     73.0   \n",
       "                         Primary Energy|Biomass|Traditional                 EJ      0.0   \n",
       "                         Primary Energy|Coal                                EJ     32.0   \n",
       "                         Primary Energy|Coal|CCUS                           EJ     16.0   \n",
       "                         Primary Energy|Coal|Unabated                       EJ     16.0   \n",
       "                         Primary Energy|Hydro                               EJ     27.0   \n",
       "                         Primary Energy|Natural Gas                         EJ     75.0   \n",
       "                         Primary Energy|Natural Gas|CCUS                    EJ     31.0   \n",
       "                         Primary Energy|Natural Gas|Unabated                EJ     44.0   \n",
       "                         Primary Energy|Nuclear                             EJ     54.0   \n",
       "                         Primary Energy|Oil                                 EJ     79.0   \n",
       "                         Primary Energy|Oil|Non-energy Use                  EJ     31.0   \n",
       "                         Primary Energy|Other renewables                    EJ     24.0   \n",
       "                         Primary Energy|Renewables (excluding traditiona... EJ    295.0   \n",
       "                         Primary Energy|Solar                               EJ     78.0   \n",
       "                         Primary Energy|Wind                                EJ     67.0   \n",
       "\n",
       "                                                                                   2050  \n",
       "model scenario    region variable                                           unit         \n",
       "IEA   Netzero2050 World  Primary Energy                                     EJ    543.0  \n",
       "                         Primary Energy|Bioenergy                           EJ    102.0  \n",
       "                         Primary Energy|Bioenergy|Gaseous                   EJ     14.0  \n",
       "                         Primary Energy|Bioenergy|Liquid                    EJ     15.0  \n",
       "                         Primary Energy|Bioenergy|Solid                     EJ     73.0  \n",
       "                         Primary Energy|Biomass|Traditional                 EJ      0.0  \n",
       "                         Primary Energy|Coal                                EJ     17.0  \n",
       "                         Primary Energy|Coal|CCUS                           EJ     14.0  \n",
       "                         Primary Energy|Coal|Unabated                       EJ      3.0  \n",
       "                         Primary Energy|Hydro                               EJ     30.0  \n",
       "                         Primary Energy|Natural Gas                         EJ     60.0  \n",
       "                         Primary Energy|Natural Gas|CCUS                    EJ     43.0  \n",
       "                         Primary Energy|Natural Gas|Unabated                EJ     17.0  \n",
       "                         Primary Energy|Nuclear                             EJ     61.0  \n",
       "                         Primary Energy|Oil                                 EJ     42.0  \n",
       "                         Primary Energy|Oil|Non-energy Use                  EJ     29.0  \n",
       "                         Primary Energy|Other renewables                    EJ     32.0  \n",
       "                         Primary Energy|Renewables (excluding traditiona... EJ    362.0  \n",
       "                         Primary Energy|Solar                               EJ    109.0  \n",
       "                         Primary Energy|Wind                                EJ     89.0  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_balances_totals.timeseries()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Append the cleaned-up data to the list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_list.append(df_balances_totals)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Electricity and Heat Supply\n",
    "\n",
    "Read the relevant section of the World_Balance sheet and cast to an **IamDataFrame**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "balances_elec_heat = pd.read_excel(file, sheet_name=\"World_Balance\", header=4, usecols=\"A:F\")[17:32]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_balances_elec_heat = pyam.IamDataFrame(balances_elec_heat, **index, region=\"World\", variable=\"Unnamed: 0\", unit=\"EJ\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Rename the fuels to IAMC-style variables similar to those used in the IPCC AR6."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "variable_mapping = {\n",
    "    \"Electricity and heat sectors\": \"Secondary Energy|Electricity and Heat\",\n",
    "    \"Coal with CCUS\": \"Coal|CCUS\",\n",
    "    \"Unabated coal\": \"Coal|Unabated\",\n",
    "    \"Natural gas with CCUS\": \"Natural Gas|CCUS\" ,\n",
    "    \"Unabated natural gas\": \"Natural Gas|Unabated\",\n",
    "}\n",
    "\n",
    "df_balances_elec_heat.rename(variable=variable_mapping, inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Prefix \"Secondary Energy|Electricity and Heat\" to variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "prefix = \"Secondary Energy|Electricity and Heat\"\n",
    "\n",
    "df_balances_elec_heat.rename(\n",
    "    variable=dict([(v, f\"{prefix}|{v}\") for v in df_balances_elec_heat.variable if v != prefix]),\n",
    "    inplace=True\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compute aggregates of timeseries data that are only given at sub-sectoral detail"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "agg_vars = [f\"Secondary Energy|Electricity and Heat|{v}\" for v in [\"Natural Gas\", \"Coal\"]]\n",
    "\n",
    "for v in agg_vars:\n",
    "    df_balances_elec_heat.aggregate(v, append=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Append the cleaned-up data to the list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_list.append(df_balances_elec_heat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Compile individual components of the reformatted data and export as csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pyam.concat(df_list)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.to_csv(\"data/iea_netzero.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}