Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Too similar file names on macos #147

Open
glatterf42 opened this issue Jan 22, 2024 · 6 comments
Open

Too similar file names on macos #147

glatterf42 opened this issue Jan 22, 2024 · 6 comments
Labels
bug Something isn't working

Comments

@glatterf42
Copy link
Member

As pointed out by @measrainsey last year and now by @ravitby, this repository currently contains both demand.csv.gz and DEMAND.csv.gz in data/test/MESSAGEix-GLOBIOM_1.1_R11_no-policy_baseline. This seems to cause git conflicts on macos systems as they don't distinguish between capitalized and non-capitalized file names.

I'm not immediately sure how we can remedy this, possibly by renaming a file, but I'm not sure if the exact names are somehow important (e.g., correspond to some model parameter and need to be exactly the way they are for the file to be found).

@glatterf42 glatterf42 added the bug Something isn't working label Jan 22, 2024
@ravitby

This comment was marked as duplicate.

@glatterf42
Copy link
Member Author

glatterf42 commented Jan 22, 2024

Thanks for the additional input. Searching for this error message, I came across this suggestion to set up a new, case-sensitive volume on macos that can host the existing code.
However, I'd consider it more elegant if we could avoid using the same name for these files. I hope @khaeru knows if such an adaptation is straightforward.

@glatterf42
Copy link
Member Author

glatterf42 commented Jan 22, 2024

There's also a git config variable that we could play around with to see if some setting satisfies our needs locally.

@khaeru
Copy link
Member

khaeru commented Jan 22, 2024

This is an unfortunate consequence of the fact that the MESSAGE GAMS code uses both a parameter named demand and a variable named DEMAND. See:
https://github.com/iiasa/message_ix/blob/693f2273eaf31aec7a9938f65ee26f98be967742/message_ix/model/MESSAGE/parameter_def.gms#L147-L159
https://github.com/iiasa/message_ix/blob/693f2273eaf31aec7a9938f65ee26f98be967742/message_ix/model/MESSAGE/data_load.gms#L75-L76
https://github.com/iiasa/message_ix/blob/693f2273eaf31aec7a9938f65ee26f98be967742/message_ix/model/MESSAGE/model_core.gms#L149-L151

The model data snapshot published on Zenodo contains data for both, and so the 'exploded' or 'unpacked' version of that file stored in this repo has both.

AFAIK, the variable DEMAND (upper case) is not relied on by any tests, currently, so I think as a mitigation the corresponding file could be safely deleted. If/when we add or migrate (from message_data) code that relies on data in this variable, we would need to find a different approach.

@measrainsey
Copy link
Contributor

Hi all, thanks for addressing this. I think I'm the source of the issue because I'm a macOS user 🥲 Like @glatterf42 said, my computer does not allow me to have files with the same spelling/name (even if they have different capitalizations). I've tried to deal with this a few times while working on my PR in message-ix-models, such as when I tried to add back the DEMAND file (in b761a93), only to later have to delete it again (in 23ae6ff). So currently no real fix on my end.

@glatterf42
Copy link
Member Author

I think we can leave this open since the merged mitigation PR is likely just a temporary fix; however, for the time being, this issue should be solved on main :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants