Migrate .model.disutility #13

khaeru · 2021-03-31T15:22:13Z

As well:

8 methods in .util used by .model.disutility.
Install GAMS using iiasa/actions workflows, parallel to Install GAMS using iiasa/actions for CI ixmp#410 and Install GAMS using iiasa/actions for CI message_ix#452.
Small tweaks and improvements to utilities, e.g. ScenarioInfo also stores duration_period when constructed off an existing scenario.

Partially addresses #4.

broadcast, copy_column, ffill, make_io, make_matched_dfs, make_source_tech, merge_data, same_node

codecov · 2021-03-31T15:31:00Z

Codecov Report

Merging #13 (17c7bec) into main (e758634) will increase coverage by 0.09%.
The diff coverage is 100.00%.

❗ Current head 17c7bec differs from pull request most recent head fe20ae7. Consider uploading reports for the commit fe20ae7 to get more accurate results

@@             Coverage Diff             @@
##             main       #13      +/-   ##
===========================================
+ Coverage   99.90%   100.00%   +0.09%     
===========================================
  Files          26        28       +2     
  Lines        1049      1318     +269     
===========================================
+ Hits         1048      1318     +270     
+ Misses          1         0       -1

Impacted Files	Coverage Δ
message_ix_models/model/disutility.py	`100.00% <100.00%> (ø)`
message_ix_models/testing.py	`100.00% <100.00%> (+1.51%)`	⬆️
message_ix_models/tests/model/test_disutility.py	`100.00% <100.00%> (ø)`
message_ix_models/tests/test_testing.py	`100.00% <100.00%> (ø)`
message_ix_models/tests/test_util.py	`100.00% <100.00%> (ø)`
message_ix_models/util/__init__.py	`100.00% <100.00%> (ø)`
message_ix_models/util/scenarioinfo.py	`100.00% <100.00%> (ø)`

khaeru · 2021-04-06T15:06:34Z

The added test_minimal() in 9c1ee66 illustrates how the MESSAGEix formulation limits setting mode-specific constraints on inter-period activity changes.

Case A

With no values for growth_activity_{lo,up}, and no difference in disutility, the following are partial results for the ACT variable:

       technology  year_act mode  lvl
               t0      2020  all  1.0
               t1      2020  all  1.0
               t1      2025  all  2.0
disutility source      2020  all  0.2
disutility source      2025  all  0.2
         t0 usage      2020   g0  0.5
         t1 usage      2020   g0  0.5
         t0 usage      2020   g1  0.5
         t1 usage      2020   g1  0.5
         t1 usage      2025   g0  1.0
         t1 usage      2025   g1  1.0

The bound_activity_{lo,up} values for 2020 set the usage of t0 and t1 output by g0 and g1 to be 0.5.
In 2025:
- The model chooses (arbitrarily; there is no difference in the objective function) to use 100% t1 and 0% t0 to provide the demand by each group g0 and g1.
- The sum across modes of "t1 usage" activity doubles from 1.0 to 2.0.

Case B

With values for growth_activity_{lo,up}, the corresponding outputs are:

       technology  year_act mode   lvl
               t0      2020  all  1.00
               t0      2025  all  1.05
               t1      2020  all  1.00
               t1      2025  all  0.95
disutility source      2020  all  0.20
disutility source      2025  all  0.20
         t0 usage      2020   g0  0.50
         t1 usage      2020   g0  0.50
         t0 usage      2020   g1  0.50
         t1 usage      2020   g1  0.50
         t0 usage      2025   g0  0.05
         t1 usage      2025   g0  0.95
         t0 usage      2025   g1  1.00

Again, there is no difference in disutility of "t0 output" and "t1 output" for either group.
In period 2025:
- The sum across modes of "t1 usage" activity decreases from 1.0 to 0.95, instead of increasing, as in the previous case.
- The sum across modes of "t0 usage" activity increases from 1.0 to 1.05, instead of decreasing from 1.0 to 0.

Case C

Now, the disutility of t1 for group g0 is increased from 0.1 to 0.2. The output is:

       technology year_act mode   lvl
               t0     2020  all  1.00
               t0     2025  all  1.05
               t1     2020  all  1.00
               t1     2025  all  0.95
disutility source     2020  all  0.20
disutility source     2025  all  0.20
         t0 usage     2020   g0  0.50
         t0 usage     2020   g1  0.50
         t1 usage     2020   g0  0.50
         t1 usage     2020   g1  0.50
         t0 usage     2025   g0  1.00
         t0 usage     2025   g1  0.05
         t1 usage     2025   g1  0.95

Group g0 is able to shift from an even split between t0 and t1 usage to 100% t0 usage. This is not what is desired.

khaeru · 2021-04-06T16:23:34Z

36de662 changes the implementation: instead of using different labels on the mode dimension for each consumer group, these are collapsed into the labels on the technology dimension.

Similar tests: cases D, E, F correspond to A, B, C respectively:

Case D

Without growth_activity_* bounds or different disutilities across t0/t1.

       technology  year_act mode  lvl
               t0      2020  all  1.0
               t1      2020  all  1.0
               t1      2025  all  2.0
disutility source      2020  all  0.2
disutility source      2025  all  0.2
usage of t0 by g0      2020  all  0.5
usage of t0 by g1      2020  all  0.5
usage of t1 by g0      2020  all  0.5
usage of t1 by g1      2020  all  0.5

usage of t1 by g0      2025  all  1.0
usage of t1 by g1      2025  all  1.0

Again the model chooses to shift both groups into 100% t1 in the second period.

Case E

With only growth_activity_* bounds. In contrast to Case B/C, these are now ±0.1 instead of ±0.1

       technology  year_vtg  year_act mode    lvl
               t0      2020      2020  all  1.000
               t0      2025      2025  all  1.000
               t1      2020      2020  all  1.000
               t1      2025      2025  all  1.000
disutility source      2020      2020  all  0.200
disutility source      2025      2025  all  0.200
usage of t0 by g0      2020      2020  all  0.500
usage of t1 by g0      2020      2020  all  0.500
usage of t0 by g1      2020      2020  all  0.500
usage of t1 by g1      2020      2020  all  0.500

usage of t0 by g0      2025      2025  all  0.705
usage of t1 by g0      2025      2025  all  0.295

usage of t0 by g1      2025      2025  all  0.295
usage of t1 by g1      2025      2025  all  0.705

usage of t1 by g1      2020      2025  all  0.100 ?!

The model chooses equal and opposite shifts for g0 and g1.
The changes of ±0.205 (absolute) or ±0.41 (relative) don't bear any obvious resemblance to the growth_activity_* constraint values.
For some reason the usage of t1 by g1 technology from vintage 2020 is active in period 2025, even though the technical_lifetime is only 5 years.

Case F

Now with unequal disutility. Because the model is already choosing (Case E) to shift g0 into t0, we increase the disutility of t0 for g0 in 2025 from 0.1 to 0.2:

       technology  year_vtg  year_act mode    lvl
               t0      2020      2020  all  1.000
               t0      2025      2025  all  0.590
               t1      2020      2020  all  1.000
               t1      2025      2025  all  1.410
disutility source      2020      2020  all  0.200
disutility source      2025      2025  all  0.230
usage of t0 by g0      2020      2020  all  0.500
usage of t1 by g0      2020      2020  all  0.500
usage of t0 by g1      2020      2020  all  0.500
usage of t1 by g1      2020      2020  all  0.500

usage of t0 by g0      2025      2025  all  0.295
usage of t1 by g0      2025      2025  all  0.705

usage of t0 by g1      2025      2025  all  0.295
usage of t1 by g1      2025      2025  all  0.705

usage of t1 by g1      2020      2025  all  0.100 ?!

This pushes g0 to use t1 in 2020.
The value appears to be at the constraint, although with the same odd value as in Case E.

khaeru · 2021-04-07T13:07:36Z

Per some discussion today in the MESSAGEix meeting:

The changes of ±0.205 (absolute) or ±0.41 (relative) don't bear any obvious resemblance to the growth_activity_* constraint values.

This is because the growth_activity_* values are annual, not per period. Opened/merged iiasa/message_ix#454 to make this more clear in the docs.

For some reason the usage of t1 by g1 technology from vintage 2020 is active in period 2025, even though the technical_lifetime is only 5 years.

It was pointed out that technical_lifetime is optional in this case, so removed it entirely. The code to set it is still visible in 1c0ef73, as well as the function dp_for(); currently unused.

Migrate .model.disutility

upstreamed in #13

khaeru added 4 commits March 31, 2021 17:07

Add 8 .util methods from message_data

eb25002

broadcast, copy_column, ffill, make_io, make_matched_dfs, make_source_tech, merge_data, same_node

Tolerate missing parameters in .util.strip_par_data()

b107483

Allow testing.bare_res(None, …)

891524a

Add .model.disutility from message_data; docs

7fe6007

khaeru added the enh New features or functionality label Mar 31, 2021

khaeru self-assigned this Mar 31, 2021

khaeru mentioned this pull request Mar 31, 2021

Migrate code from message_data #4

Open

9 tasks

khaeru added 8 commits April 1, 2021 13:45

Remove unused scenario arg to .disutility.get_spec(); annotate types

8c99776

Include input and output commodities in .disutility.get_spec()

3d0b8fd

Improve checks in .util.broadcast()

b70073c

Add tests of .model.disutility

57a4099

Sort imports; satisfy mypy

3309f02

Install GAMS license on GHA runner

f877aab

Reflow docstrings in .util

d0b3a77

Use iiasa/actions to set up GAMS

9058605

khaeru force-pushed the feature/disutility branch from ab579b5 to 9058605 Compare April 1, 2021 12:52

khaeru added 5 commits April 1, 2021 17:57

Update test_disutility.test_get_spec()

de4fda6

Ensure unique indices from .util.broadcast and .util.make_matched_dfs

89bc180

Store duration_period when ScenarioInfo() is initialized from a Scenario

55b06c4

Set technical_lifetime for 1-period technologies using duration_period

5d75271

Expand test_disutility.test_minimal

9c1ee66

khaeru force-pushed the feature/disutility branch from 3c4864d to 9c1ee66 Compare April 6, 2021 14:45

khaeru added 3 commits April 6, 2021 17:54

Collapse consumer groups into "technology" dimension, not "mode"

36de662

Adjust test_disutility.test_minimal

4ee7446

Remove unused imports

5ab9cad

khaeru mentioned this pull request Apr 7, 2021

Expand docs of dynamic constraint parameters iiasa/message_ix#454

Merged

4 tasks

Make technical_lifetime optional for .util.make_source_tech()

1c0ef73

khaeru added 8 commits April 7, 2021 12:40

Remove technical_lifetime for disutility source and usage technologies

5ea01f8

Drop extra columns in .util.{broadcast,make_matched_dfs}

37ad525

Ensure disutility is unitless

745d3aa

Clean up test_disutility.test_minimal

9469dba

Expand documentation

af8c680

Sort imports in test_disutility.py

bb34e53

Add #13 to doc/whatsnew

37e68c8

Add test coverage of several stray lines

fe20ae7

khaeru merged commit 567c1ce into main Apr 7, 2021

khaeru deleted the feature/disutility branch April 7, 2021 13:07

glatterf42 pushed a commit that referenced this pull request Jan 24, 2023

Add #13 to doc/whatsnew

f882864

glatterf42 pushed a commit that referenced this pull request Jan 24, 2023

Merge pull request #13 from iiasa/feature/disutility

4835a76

Migrate .model.disutility

khaeru added a commit that referenced this pull request Jul 29, 2024

Remove .model.disutility & docs; adjust imports

11d60f4

upstreamed in #13

khaeru added a commit that referenced this pull request Jul 29, 2024

Adjust imports for code upstreamed in #13

e242f8b

khaeru added a commit that referenced this pull request Jul 30, 2024

Remove .model.disutility & docs; adjust imports

d287834

upstreamed in #13

khaeru added a commit that referenced this pull request Jul 30, 2024

Adjust imports for code upstreamed in #13

6e77cfd

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Migrate .model.disutility #13

Migrate .model.disutility #13

khaeru commented Mar 31, 2021 •

edited

Loading

codecov bot commented Mar 31, 2021 •

edited

Loading

khaeru commented Apr 6, 2021 •

edited

Loading

khaeru commented Apr 6, 2021

khaeru commented Apr 7, 2021

Migrate .model.disutility #13

Migrate .model.disutility #13

Conversation

khaeru commented Mar 31, 2021 • edited Loading

codecov bot commented Mar 31, 2021 • edited Loading

Codecov Report

khaeru commented Apr 6, 2021 • edited Loading

Case A

Case B

Case C

khaeru commented Apr 6, 2021

Case D

Case E

Case F

khaeru commented Apr 7, 2021

khaeru commented Mar 31, 2021 •

edited

Loading

codecov bot commented Mar 31, 2021 •

edited

Loading

khaeru commented Apr 6, 2021 •

edited

Loading