preprocess_data.py script is not compatible with any released version of the code.

[tniet]

The preprocessing of data and creation of the MODEX sets is faster with the python script than having GLPK do it during model generation. Currently, the preprocessing script is not compatible with the fast code (or is the fast code not compatible with the script?). The fast code should be released with the fastest way of running the model. I propose we make the fast code script compatible, and provide a short intro to the fast code with explanation on how to run the script to use it.

[abhishek0208]

preprocess_data.py works with a modified version of osemosys_fast. Specifically, it requires that the sets of sets (e.g. MODExTECHNOLOGY*) are declared but not calculated in the model file. I'm not sure how best to approach this issue of compatibility of having the GNU Mathprog model file and Python preprocessing file interact. Could Amply help with this?

[willu47]

Hi @abhishek0208 - thanks for the bug fixes and explanation. Perhaps just updating the docstring for the pre-processing script with more detailed instructions on how to make an osemosys model "script compatible" would be fine?

[willu47]

@abhishek0208 - @JoPapp mentioned that it is difficult to understand which flag (otoole or momani) to use with a standard datafile (e.g. not generated by otoole or momani). What does this flag do?

[willu47]

Also, in the fast code @JoPapp mentioned that there is an extra subset defined - MODEperTechnologyEmission which does not appear in the preprocessing script.

[willu47]

The other option is that we pre-add the set definitions to all of the model codes - so that the long, short and fast versions are all preprocessing-compatible.

[JoPapp]

In the preprocessing script Line 291-292 is mentioned/used MODEperTechnologyEmission. This set is been generated only if there are emissions accounted in the osemosys model. However, glpk will not run since the preprocessing script generates the MODEperTechnologyEmission for a technology without having any value and the osemosys_Fast_Code has a constraint Line 225-226 that emissionactivityratio of technology needs to be <>0. So there is a conflict

[abhishek0208]

@abhishek0208 - @JoPapp mentioned that it is difficult to understand which flag (otoole or momani) to use with a standard datafile (e.g. not generated by otoole or momani). What does this flag do?

@willu47 the otoole/momani flag is to specify the format of the data file to be (pre-)processed. They could be called long and wide instead. For now I'll update the readme.