-
Notifications
You must be signed in to change notification settings - Fork 204
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix to_checksums
with None
values in dicts and recursion
#4159
base: develop
Are you sure you want to change the base?
Conversation
I traced the failure to a difference in YEB and EB formats and the validator which seemingly has a bug: The
The difference is:
I.e. the EC correctly uses tuples while the YEB uses lists. However actually both are wrong and need to be converted because alternative checksums must be specified as a tuple: easybuild-framework/easybuild/tools/filetools.py Line 1291 in 8ba299f
However the current definition of the checksums type is:
I.e. the we wrongly allow a list where we expect a tuple. |
9e74083
to
c035dea
Compare
75b74ba
to
e8ef6d0
Compare
@Flamefire This needs another sync with |
e8ef6d0
to
c413a74
Compare
c413a74
to
8249d66
Compare
8249d66
to
3c6c5a2
Compare
The `None` case was missed and due to the unrestricted `tuple` elem_type it may return valid for actually invalid entries. So restrict that beeing overly cautious so it may wrongly return invalid. But in that case the conversion function will be called which can do more elaborate verification. Add test checking for None in checksums.
Having a `'src': None` entry in a dict for checksums is as valid as having a `None` entry directly in the list. However the current function didn't handle it and crashed. Fix that as well as a few corner cases especially in the recursive case by introducing a new function for handling checksum entries in the checksum list and limiting the recursiveness. Fixes easybuilders#4142
feda9b2
to
8335f25
Compare
Having a
'src': None
entry in a dict for checksums is as valid as having aNone
entry directly in the list. However the current function didn't handle it and crashed.Fix that as well as a few corner cases especially in the recursive case by introducing a new function for handling checksum entries in the checksum list and limiting the recursiveness.
It will now throw an
EasyBuildError
similar to e.g.to_dependency
instead of trying to continue and fail at a random other place.I extended the testcases to cover the original issue (None as a value in a dict) and some more correct and error cases
Please carefully review and check if you can come up with any valid or invalid
checksums
entry that might break using the new function. Especially regarding the YEB compatibility this was pretty hard as for lists of strings you cannot know if those are 2 checksums without a type that should both match or a checksum and type. In EC-files (Python) one can use a tuple for that but then again it is hard to tell if that tuple is a checksum and type or 2 alternative checksums.Fixes #4142
checksums
easyconfig parameter easybuild-docs#104