Change technology inputs and replace solar + wind technologies in `tools.costs` #206

measrainsey · 2024-07-16T15:48:05Z

Make changes to the energy module input technologies

All changes made in this PR are contained to the input CSVs, so no changes to the code. The main changes are:

Change inputs for some storage, CSP, hydrogen, and industry technologies (more details below)
Replace solar and wind technologies with new ones (more details below)

Change inputs for some storage, CSP, hydrogen, and industry technologies

cc @volker-krey

The following changes were made to the input CSVs:

Update stor_ppl to use the same scenario categories as solar_pv_ppl
Update csp_sm1_ppl and csp_sm3_ppl to borrow from solar_th_ppl as starting point
For many _i and hpl technologies, change the mapping of WEO technologies
Change hydrogen technology mapping to fuel_cell in WEO

Replace existing wind and solar technologies with new technologies

cc @OFR-IIASA

As part of the SSP update process, solar and wind technologies are being replaced with new (more) technologies. I have copied over the inputs/assumptions from the previous technologies and used them for the new technologies.

solar_pv_ppl has been replaced with:

solar_res_hist_2005
solar_res_hist_2010
solar_res_hist_2015
solar_res_hist_2020
solar_res1
solar_res2
solar_res3
solar_res4
solar_res5
solar_res6
solar_res7
solar_res8

wind_ppl has been replaced with:

wind_res_hist_2005
wind_res_hist_2010
wind_res_hist_2015
wind_res_hist_2020
wind_res1
wind_res2
wind_res3
wind_res4

wind_ppf has been replaced with:

wind_ref_hist_2005
wind_ref_hist_2010
wind_ref_hist_2015
wind_ref_hist_2020
wind_ref1
wind_ref2
wind_ref3
wind_ref4
wind_ref5

How to review

For @khaeru and/or @glatterf42 : Read the diff and note that the CI checks all pass.

PR checklist

~~Add or expand tests;~~ Modified tests; coverage checks both ✅
Add, expand, or update documentation.
Update doc/whatsnew.

khaeru · 2024-07-19T13:46:17Z

@measrainsey here I'd suggest "Update branch" → "Update with rebase". Should be no conflicts.

codecov · 2024-07-24T09:31:27Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 65.5%. Comparing base (f6de7bb) to head (1788b18).
Report is 21 commits behind head on main.

Additional details and impacted files

@@          Coverage Diff          @@
##            main    #206   +/-   ##
=====================================
  Coverage   65.5%   65.5%           
=====================================
  Files        204     204           
  Lines      15822   15822           
=====================================
  Hits       10378   10378           
  Misses      5444    5444

Files with missing lines	Coverage Δ
message_ix_models/tests/model/test_bare.py	`100.0% <ø> (ø)`
message_ix_models/tests/model/test_structure.py	`100.0% <100.0%> (ø)`
message_ix_models/tests/tools/costs/test_decay.py	`100.0% <ø> (ø)`
...ge_ix_models/tests/tools/costs/test_projections.py	`100.0% <ø> (ø)`
...tests/tools/costs/test_regional_differentiation.py	`100.0% <ø> (ø)`

measrainsey · 2024-07-24T09:43:17Z

Sorry, I forgot to check the tests 🤦🏻‍♀️ Turns out they were failing because tests.costs was still checking for the previous solar and wind technologies. I also had to add some of the new technologies to data/technology.yaml, which is used in test_bare_res() in tests.costs, otherwise that test would fail.

message-ix-models/message_ix_models/data/technology.yaml

Lines 594 to 606 in 48454fd

    
           solar_res8: 
        
             name: solar_res8 
        
             description: Maximum solar electricity potential 8 
        
             type: secondary 
        
             sector: electricity 
        
             output: [electr, secondary] 
        
           solar_res_hist: 
        
             name: solar_res_hist 
        
             description: Currently installed solar electricity capacity 
        
             type: secondary 
        
             sector: electricity 
        
             output: [electr, secondary]

Because I added these new technologies, I also had to update some of the tests in tests.model to reflect that the number of technologies have changed from 377 to 381.

All checks passing now, but let me know if my changes might be problematic.

measrainsey · 2024-07-31T13:25:40Z

Just a note that I added some more technologies: 2005, 2010, 2015, and 2025 bins for the _hist solar and wind technologies. Tests and the technology.yaml have been updated to reflect this as well.

glatterf42

One small question, otherwise, this looks good to me (after rebasing onto current main :)

glatterf42 · 2024-09-11T05:49:25Z

doc/whatsnew.rst

@@ -16,6 +18,7 @@ v2024.8.6
 - Add :doc:`/project/edits` project code and documentation (:pull:`204`).
 - Reduce log verbosity of :func:`.apply_spec` (:pull:`202`).
 - Fix and update :doc:`/api/tools-costs` (:pull:`186`, :pull:`187`, :pull:`190`, :pull:`195`).
+- Made fixes and updates to :doc:`/api/tools-costs` (:pull:`186`, :pull:`187`, :pull:`190`, :pull:`195`, :pull:`206`).


Why do we get this new line instead of expanding the one above it (using "Fix und update ...")?

Thanks for the catch :) Think I made an error within doc/whatsnew along the way by adding this PR #206 to the v2024.8.6 release instead of under "Next release". I deleted this entry under v2024.8.6, but this PR is documented under "Next release".

We also have this line:

message-ix-models/.gitattributes

Lines 1 to 2 in 32cb2fd

# Reduce the number of merge conflicts

doc/whatsnew.rst merge=union

—but it is still incumbent on PR authors to keep the entries in the right order.

Thanks all - after a few mistakes I think I sorted the doc/whatsnew issues. Took a few commits to get there but hope that's fine.

For the subsequent PRs #221, #227, and #222, I rebased and dropped all the other commits related to doc/whatsnew because they were creating some clutter. Then just did one final edit to doc/whatsnew for each PR.

khaeru · 2024-09-26T09:04:30Z

@measrainsey @glatterf42 from my POV this is (a) up to date with rebase and (b) approved. Can we now merge?

measrainsey · 2024-09-26T09:09:49Z

@measrainsey @glatterf42 from my POV this is (a) up to date with rebase and (b) approved. Can we now merge?

Yep - if all good with everyone, I'm happy to merge. Thanks!

…y technologies Main changes are: - update `stor_ppl` to use the same scenario categories as `solar_pv_ppl` - update `csp_sm1_ppl` and `csp_sm3_ppl` to borrow from `solar_th_ppl` as starting point - for many `_i` and `hpl` technologies, change mapping of WEO technologies - change hydrogen to be mapped to `fuel_cell` in WEO

Copied over the values (base year mapping, first year, cost reduction rates, and scenario mappings) from solar_pv_ppl for new solar technologies: - solar_res1 - solar_res2 - solar_res3 - solar_res4 - solar_res5 - solar_res6 - solar_res7 - solar_res8 - solar_res_hist

Copied over input data for wind_ppl and added new technologies: - wind_res1 - wind_res2 - wind_res3 - wind_res4 - wind_res_hist

Copied over input data for wind_ppf and replaced with the following new technologies: - wind_ref1 - wind_ref2 - wind_ref3 - wind_ref4 - wind_ref5 - wind_ref_hist

- Replace `solar_pv_ppl` with `solar_res1` - Replace `wind_ppl` with `wind_res1`

…ata/technology.yaml

glatterf42 · 2024-09-26T09:24:56Z

Rebased to resolve conflicts in doc/whatsnew, but should be good to go :)

measrainsey self-assigned this Jul 16, 2024

Base automatically changed from costs/fom-vintage to main July 19, 2024 13:45

measrainsey force-pushed the costs/new-inputs branch 3 times, most recently from 303fbf8 to 48454fd Compare July 24, 2024 08:39

measrainsey force-pushed the costs/new-inputs branch from 7fcefdc to 1f166a7 Compare August 1, 2024 10:30

measrainsey force-pushed the costs/new-inputs branch from 7373735 to 6c37f66 Compare August 13, 2024 09:02

measrainsey mentioned this pull request Aug 29, 2024

Reconfigure usage of technology modules in tools.costs #221

Merged

2 tasks

khaeru added the costs `.tools.costs`/cost data preparation label Aug 29, 2024

measrainsey force-pushed the costs/new-inputs branch from b3af303 to f3d379a Compare September 2, 2024 11:43

glatterf42 approved these changes Sep 11, 2024

View reviewed changes

measrainsey force-pushed the costs/new-inputs branch 2 times, most recently from a727124 to 621611d Compare September 11, 2024 09:16

adrivinca force-pushed the costs/new-inputs branch from 621611d to faa43b7 Compare September 25, 2024 12:27

measrainsey force-pushed the costs/new-inputs branch from faa43b7 to 00d4120 Compare September 26, 2024 07:54

khaeru mentioned this pull request Sep 26, 2024

Collect pieces for SSP 2024/ScenarioMIP workflows #234

Open

measrainsey added 8 commits September 26, 2024 11:21

Replace wind_ppl with new onshore wind technologies

54493fa

Copied over input data for wind_ppl and added new technologies: - wind_res1 - wind_res2 - wind_res3 - wind_res4 - wind_res_hist

Replace wind_ppf with new offshore wind technologies

2c11820

Copied over input data for wind_ppf and replaced with the following new technologies: - wind_ref1 - wind_ref2 - wind_ref3 - wind_ref4 - wind_ref5 - wind_ref_hist

Sort all energy module input CSVs alphabetically

3e7d607

Update doc/whatsnew

9deda88

Replace old wind and solar technologies in tests

5e7e25d

- Replace `solar_pv_ppl` with `solar_res1` - Replace `wind_ppl` with `wind_res1`

Add solar_res8, solar_res_hist, wind_res_hist, and wind_ref_hist to d…

055aaba

…ata/technology.yaml

measrainsey added 11 commits September 26, 2024 11:21

Change number of technologies from 377 to 381 in tests.model

82e7153

Add year bins for _res_hist technologies in input CSVs

3586163

Add binned technologies to data/technology.yaml

7a8e437

Update number of technologies in tests.model to 390

16b9a5f

Add PR number to doc/whatsnew

25ed322

Remove wind_ppf technology from input

1428a7e

Move whatsnew to next release

96da4e0

Update doc/whatsnew

9ae6cda

Remove entry from previous release in doc/whatsnew

aaccd3f

Remove duplicate entry in doc/whatsnew

74a8051

Update doc/whatsnew again after rebasing

1788b18

glatterf42 force-pushed the costs/new-inputs branch from 00d4120 to 1788b18 Compare September 26, 2024 09:24

khaeru merged commit 4d8ec92 into main Sep 26, 2024
26 checks passed

khaeru deleted the costs/new-inputs branch September 26, 2024 10:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Change technology inputs and replace solar + wind technologies in `tools.costs` #206

Change technology inputs and replace solar + wind technologies in `tools.costs` #206

measrainsey commented Jul 16, 2024 •

edited

Loading

khaeru commented Jul 19, 2024

codecov bot commented Jul 24, 2024 •

edited

Loading

measrainsey commented Jul 24, 2024 •

edited

Loading

measrainsey commented Jul 31, 2024

glatterf42 left a comment

glatterf42 Sep 11, 2024

measrainsey Sep 11, 2024

khaeru Sep 11, 2024

measrainsey Sep 11, 2024

khaeru commented Sep 26, 2024

measrainsey commented Sep 26, 2024

glatterf42 commented Sep 26, 2024

	# Reduce the number of merge conflicts
	doc/whatsnew.rst merge=union

Change technology inputs and replace solar + wind technologies in tools.costs #206

Change technology inputs and replace solar + wind technologies in tools.costs #206

Conversation

measrainsey commented Jul 16, 2024 • edited Loading

Make changes to the energy module input technologies

Change inputs for some storage, CSP, hydrogen, and industry technologies

Replace existing wind and solar technologies with new technologies

How to review

PR checklist

khaeru commented Jul 19, 2024

codecov bot commented Jul 24, 2024 • edited Loading

Codecov Report

measrainsey commented Jul 24, 2024 • edited Loading

measrainsey commented Jul 31, 2024

glatterf42 left a comment

Choose a reason for hiding this comment

glatterf42 Sep 11, 2024

Choose a reason for hiding this comment

measrainsey Sep 11, 2024

Choose a reason for hiding this comment

khaeru Sep 11, 2024

Choose a reason for hiding this comment

measrainsey Sep 11, 2024

Choose a reason for hiding this comment

khaeru commented Sep 26, 2024

measrainsey commented Sep 26, 2024

glatterf42 commented Sep 26, 2024

Change technology inputs and replace solar + wind technologies in `tools.costs` #206

Change technology inputs and replace solar + wind technologies in `tools.costs` #206

measrainsey commented Jul 16, 2024 •

edited

Loading

codecov bot commented Jul 24, 2024 •

edited

Loading

measrainsey commented Jul 24, 2024 •

edited

Loading