Add technoeconomic cost projections and related data #99

The following changes were made: - renamed and sorted dicts - used pandas chaining when possible - replaced nested loop with itertools.product() - edited docstrings style - renamed data files - added source information when possible

Had to delete the _autosummary and _build folders and rebuild again for the sphinx build to work properly The files in the _autosummary kept looking for iea.costs module

The script does the following: - Read in raw GDP data for SSP1, SSP2, and SSP3 scenarios - Calculate regional GDP ratios - Apply linear regression using regional GDP regional ratios and regional cost ratios

…ariables into inputs

…arios

This updated methodology looks like following: 1. Apply learning rates to NAM region investment costs 2. Use regionally differentiated GDP ratios and cost ratios to develop a linear model (predict cost ratios based on GDP ratio) 3. Calculate adjusted (annual) regionally differentiated cost ratio 4. Project investment costs in all regions using the new adjusted cost ratios 5. Perform polynomial regression on projections from (4) 6. Use regression splines to project investment costs (if NAM, use learning projections from (4); for all other regions use splines results) 7. Use fixed-to-investment cost ratios to calculate fixed O&M costs based on (6)

…scenario

…gion

- Update methodology to read in mapping of technologies and base year costs - Removed manual calculations of base year and reference region costs - Allow for specification of node/spatial resolution, base year, and reference region - Change inflation rate - Replace missing values in WEO data with medians of technology

- Node (R11, R12, or R20) - Base region (by default NAM for each) - Base year - SSP scenario version (updated/review or previous) - SSP scenario - Projection method (learning, GDP adjusted, or convergence with splines) - Convergence year

…cted

…atio

The new calculation introduces a different linear regression where each region-technology's base year cost ratio is linearly regressed to a hypothetical (where GDP ratio == 1, cost ratio == 1).

This is a very rough implementation of the LED (low energy demand) scenario. All assumptions need serious and careful verification, as I mainly guessed values for now. I added values of cost reduction rates and assigned learning rate categories for each technology under an LED scenario (see `cost_reduction_rates.csv` and `technology_learning_rates.csv`). I assigned SSP2 population and GDP per capita data to the LED scenario (see `gdp.py`). Finally, I added the LED scenario to the projections functions (see `projections.py`).

Remove unused code

For now, only projecting the technologies that can be mapped to the base set of technologies

…ings

Also cherry-pick 80464da to satisfy mypy.

1. Renamed input CSVs 2. Renamed weo.py script to regional_differentiation.py

…eg_diff_technology

- Add Intratec regional differentiation - Make assumptions for missing regional differentiation mappings - Make assumptions for cost reduction rates and scenarios - Add fix-to-inv cost ratios as input

Use same cost reduction rates as SSP2 except for VRE and coal

This is more of a quick fix, the duplicates are most likely happening during some merging. In the `learning.py` script, adding the `drop_duplicates()` got rid of some of the duplicates, but not all of them. I think the final duplication is happening in `create_message_outputs()`, as adding `drop_duplicates()` there seemed to get rid of them all.

…ustment Tests were taking too long (linearly regressing GDP data), so I've added the option to filter by scenario during the GDP regression process as well. Code is a little redundant now and clunky but to be fixed later.

`.pivot()` was crashing for some reason (saying there were duplicates in the index), but `.pivot_table()` from `pandas` seems to work just fine

- This is possible because the assertions are the same in both cases. - Use get_codes() instead of hard-coded lists of node IDs. - Use f-string for ref_region argument.

Also: Xfail some cases.

- Improve typing - Set context.model.regions in test

- Store a "name" keyword argument, if given.

Also set test_context.model.regions.

- Adjust tests. - Adjust usage in demo.py; expand comments. - Add Config.check() for validation of settings. - Use dataclass default Config.__repr__() to show contents.

- Use f-strings for output. - Simplify/align construction of scen, scen_vers variables.

- Also use relative imports within .tools.costs

- .gdp.adjust_cost_ratios_with_gdp(). - Also re-use .projections._maybe_query_*() functions instead of duplicated logic. - .learning.project_ref_region_inv_costs_using_learning_rates() - .regional_differentiation.apply_regional_differentiation() - Also rely on Config logic for node/ref_region. Adjust tests to align.

- Remove "projections" class. - Adjust tests, demo.py. - Add "-> None" to ensure .tests.tools.costs functions are type checked.

Log output can be handled (redirected, tested, filtered) more precisely than standard output.

- Move common statements before or after if-blocks. - Specify required file format, location, and name in docstring of get_raw_technology_mapping().

- Retrieve World Bank data via IAM-units. - Adjust test.

- Deduplicate code for module="energy" and module="materials". - Simplify set logic for assertions about returned data. - Correct rtol=3 (factor of 3) to rtol=5e-2 (± 5%) in test of apply_splines_to_convergence(); expand to all technologies.

- Reference Config.base_year. - Use built-in get_codes() to get the year set.

- Generate the `seq_years` variable via Config. - Reuse ScenarioInfo.y0 semantics for FIRST_MODEL_YEAR. - Document difference between "LAST_MODEL_YEAR"/Config.final_year and the actual final model year appearing in most MESSAGEix-GLOBIOM scenarios.

- Filter year_* data using Config properties.

- Avoid product() and manual handling of group keys.

- Use numpy.polynomial instead of scikits-learn: lower-level, faster, avoids a new dependency, identical output. - Improve performance: - Fit and predict in the same step. - Chain all pandas operations in the function.

- First line of docstring ends with a period. - Incorporate pre-function comments into docstring, so they are visible in the built documentation. - Ensure ReST list items are wrapped.

- Use .eval(…) with direct formulae instead of 2-stage linregress()/predict on 2 data points per group. - Vectorize calculations for performance. - Chain pandas operations. - Remove .gdp.default_ref_region(); this predates Config, is now handled by Config.

This file was initially created to subset the large SSP data file. But with the new SSP data hhandler, I think this script is no longer needed.

- For projecting costs of reference region, used words like `decay` or `cost reduction` instead - Renamed `learning.py` to `decay.py` and also the corresponding test file - Changed `learning` projection method to `constant` instead

Namely, that `splines` have been removed and mentions of the term `learning` have been changed to `decay`, `cost reduction`, and `constant` (where appropriate)

- Explain more about the costs module functionality and different projection methods - Explain how to add additional technology modules - Give example of how to use code

…ll non-reference regions

- Start new sentences on new lines. - Use "-" consistently for bullet lists (don't mix "*" and "-"). - Ensure a blank line before bullet lists. - Use the ReST :file:`...` role for file names and paths.

- Shorten title for display. - Explain terms used. - Correct spelling and grammar. - Update docstrings for .costs.Config attributes and refer to these, instead of duplicating descriptions in text. - Update headings. - Use quotes for values appearing in code or data files.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add technoeconomic cost projections and related data #99

Add technoeconomic cost projections and related data #99

Commits on Apr 9, 2024