Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Validating empty array against spec should show better error #317

Open
rly opened this issue Mar 19, 2020 · 3 comments · May be fixed by #319
Open

Validating empty array against spec should show better error #317

rly opened this issue Mar 19, 2020 · 3 comments · May be fixed by #319
Labels
category: bug errors in the code or code behavior topic: validator issues related to validation of files

Comments

@rly
Copy link
Contributor

rly commented Mar 19, 2020 •
edited
Loading

If an attribute spec specifies a dtype text with or without a shape, and an empty numpy array is validated against that spec, the error returned is: IndexError: index 0 is out of bounds for axis 0 with size 0. This is not very helpful.

Is an empty array allowed for shape: None (1D array with any length) ?

To clarify, what should be the correct validation behavior for the following cases:

  1. Spec specifies shape: None and dtype: float and test value is empty array with type float -- ShapeError?
  2. Spec specifies shape: None and dtype: text and test value is empty array with type float -- ShapeError and DtypeError?
  3. Spec specifies shape: None and dtype: text and test value is empty list -- ShapeError?
  4. Spec specifies scalar and dtype: text and test value is empty array/list -- ShapeError?
  5. Spec specifies scalar and dtype: text and test value is empty string -- OK?
@rly rly added the category: bug errors in the code or code behavior label Mar 19, 2020
@rly rly mentioned this issue Mar 19, 2020
Open
@oruebel
Copy link
Contributor

oruebel commented Mar 19, 2020

  • Spec specifies shape: None and dtype: float and test value is empty array with type float -- ShapeError?

I think this would be a warning. shape: None does not seem to imply a minimum size, if it does then we would need a way to say that a dataset may be empty.

  • Spec specifies shape: None and dtype: text and test value is empty array with type float -- ShapeError and DtypeError?

This seems to be clearly a DtypeError first potentially combined with a warning that the dataset is empty.

  • Spec specifies shape: None and dtype: text and test value is empty list -- ShapeError?

Do you mean you don't have a dtype for the data in this case? I would issue warnings here.

  • Spec specifies scalar and dtype: text and test value is empty array/list -- ShapeError?

A scalar cannot be an array/list so a ShapeError seems appropriate.

  • Spec specifies scalar and dtype: text and test value is empty string -- OK?

Empty string is a valid string. I would issue a warning but not an error.

@rly
Copy link
Contributor Author

rly commented Mar 19, 2020

Should the official pynwb validator issue these warnings for empty strings and empty datasets? Or should this instead be bundled into the best practices inspector? The validator currently does not issue any warnings.

@oruebel
Copy link
Contributor

oruebel commented Mar 19, 2020

Since these are generic issues (i.e. issues that are not specific to the NWB schema), I think it would be fine to have such warnings be issued by the validator. It may be worthwhile to have this behavior configurable to enable/disable warnings.

@rly rly linked a pull request Mar 24, 2020 that will close this issue
Draft
@rly rly added the topic: validator issues related to validation of files label Nov 25, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
category: bug errors in the code or code behavior topic: validator issues related to validation of files
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix bug on gettype on empty array, add validator tests hdmf-dev/hdmf
2 participants
@rly @oruebel