Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sh:minInclusive of cube constraint set at higher value than lowest in dataset #1510

Open
1 of 4 tasks
AdMSilvan opened this issue May 7, 2024 · 6 comments
Open
1 of 4 tasks
Assignees
Labels
🐛 bug Something isn't working

Comments

@AdMSilvan
Copy link

AdMSilvan commented May 7, 2024

Describe the bug
When validating some datasets against their cube constraint, many errors are reported due to sh:minInclusive being set to a value higher than some of the values contained in the dataset.

Affected functionalities (all that apply)

  • CSV Mapping
  • Transformation
  • Publishing
  • Other

Relevant links

https://culture.ld.admin.ch/sfa/StateAccounts_Office/5/observation/OC1-O3-1951
https://culture.ld.admin.ch/sfa/StateAccounts_Function/7/shape/

To Reproduce
Steps to reproduce the behavior:
validate cubes with schema:amount 0, where minInclusive is not set to 0

Presumably CubeCreator still creates cubes with these constraint shapes if given a dataset with values containing 0.
This does not seem to happen in every case, since in this dataset the shape was set correctly: https://culture.ld.admin.ch/sfa/StateAccounts_Domain/11/shape/

Expected behavior
sh:minInclusive should be set to 0 if this is the lowest value in the data.

Additional context
The cause is unclear. At first I assumed it could be caused by 0 not being taken as a proper value and setting the min to the next lowest number, but this example shows that the value set for the min is not the lowest:
Constraint-Shape with min value for amount: https://culture.ld.admin.ch/sfa/StateAccounts_Function/7/shape/#blank-2
Lowest values in dataset: https://s.zazuko.com/27ykjNn

@AdMSilvan AdMSilvan added the 🐛 bug Something isn't working label May 7, 2024
@AdMSilvan
Copy link
Author

@Rdataflow discovered this issue so he may be able to add more context.

@Rdataflow
Copy link
Contributor

Rdataflow commented May 7, 2024

@AdMSilvan thank you for reporting this one

important amendment:

Violation: "Value is not less than or equal to 7952" at focus node <https://energy.ld.admin.ch/sfoe/bfe_ogd100_kennzahlensharedmobility/2/observation/kennzahlen_sharedmobility/Baden%C2%A0%E2%80%93%C2%A0Brugg/2021-12-16T09%3A40%3A00> with path <https://energy.ld.admin.ch/sfoe/bfe_ogd100_kennzahlensharedmobility/numberOfEScooters> (MaxInclusiveConstraintComponent of source: _:b89_b90_b90_genid2d701d2c81431d48758eee86f5ea15a8442d1895464E9C9B88EE7E7F05C70132E99C1)

how to solve is already documented in https://cube.link/#example-literals-dimensions and thus would look like

[
    sh:path ex:literalDimension;
    sh:nodeKind sh:Literal;
    sh:or([
        sh:datatype xsd:string ; sh:minInclusive 0 ; sh:maxInclusive 100
    ], [
        sh:datatype cube:Undefined
    ])
]

@tpluscode how to test and DOD

  • consistent sh:datatype including range (sh:minInclusive and sh:maxInclusive) between observationConstraint and actual observations
  • validation of observations no longer violates on minInclusive or maxInclusive after republishing a cube... neither for 0 (zero) values ... nor for ""^^cube:Undefined (undefined)

@AdMSilvan
Copy link
Author

AdMSilvan commented May 7, 2024

@Rdataflow I believe the issue with cube:Undefined was already caught earlier and is documented in this bug report #1470 which is why I left it out of this one. The cause also seems to be different.

Thank you for your addition, I should have marked them as related to begin with.

@tpluscode
Copy link
Collaborator

@AdMSilvan did we agree that, like #1470, we'll keep this issue out of current sprint?

@AdMSilvan
Copy link
Author

@tpluscode in the visualize call today we were able to speak to Martina and we thought it would be best to give estimates for the three issues I added in the gitlab page, so that we could potentially fit them into this or the upcoming sprint as well, so we can avoid organizational overhead.

@tpluscode
Copy link
Collaborator

Understood

That said, I believe this is exactly the same as #1470

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐛 bug Something isn't working
Projects
Status: Backlog
Development

No branches or pull requests

3 participants