Schema error: constraint "maxLength" is not supported by type "any"

[whalekeykeeper]

Hi,
after our project failed the automatic validation, according to the properties of constraints on the Table Schema section in Frictionless Data Specs, I replaced "max-length" with "maxLength", "min-length" with "minLength". Then I tried to validate our data again but manually this time with frictionless validate csv/datapackage.json, still, got many error messages, for example:

schema-error  The data source could not be successfully described by the invalid Table Schema: constraint "maxLength" is not supported by type "any"  
schema-error  The data source could not be successfully described by the invalid Table Schema: constraint "minLength" is not supported by type "any" 
schema-error  The data source could not be successfully described by the invalid Table Schema: constraint "pattern" is not supported by type "any"   
schema-error  The data source could not be successfully described by the invalid Table Schema: constraint "type" is not supported by type "any"

I am using the frictionless version 3.45.5 on macOs 10.15.4.

It is said A constraints descriptor MUST be a JSON object and MAY contain one or more of the following properties with a table including properties like minLength, maxLength, pattern. Meanwhile, on the same webpage, it is said that for any, Any type or format is accepted. When converting from physical to logical representation, the behaviour should be similar to String field type.

I have three questions:

1. Why some constraints are not supported by type "any"?
2. Can constrains has a type property? For example, "constraints": {"required": true,"type": "http://www.w3.org/2001/XMLSchema#date"}.
3. The error message did not return any information about locations or even any file names. In another word, it is not quite useful. Could this be Improved?

#readchina/ReadAct#447

[duncdrum]

so this is at minimum a breaking change in behaviour between the different versions, that wasn't part of the release notes. I still think this is a bug, because there is nothing preventing any from being constrained. Just because the data type is more general the string doesn't mean setting a max-length is inherently non-sensical.

I m not sure why this has been moved to discussion, but didn't warrant a response? Is the documentation outdated and wrong, is the new behaviour intended, or is this an implemementation bug?

[roll]

Hi,

can you please post the whole Table Schema? Also, the current frictionless version is 4.9. Can you please try this version and let me know whether it still doesn't work?

[duncdrum]

frictionless frictionless-4.9.1 has the same behaviour:

===========  ==========================================================================
field-error  Field is not valid: constraint "max-length" is not supported by type "any"
===========  ==========================================================================

Link to datapackage

latest CI run

[roll]

I think the problem is that this schema - https://github.com/readchina/ReadAct/blob/70aee05b3431f3bac5a448ecd6a04c4bcb0b772f/csv/schema/Person.json - doesn't set data types and based on the spec - https://specs.frictionlessdata.io/table-schema/#constraints - any type (which is the default) cannot support the constraints provided.

The fix should be adding data types:

      "name": "person_id",
      "type": "string", # ADDED
      "constraints": {
        "required": true,
        "unique": false,
        "max-length": 6,
        "pattern": "AG[0-9]{4}"
      }

[duncdrum]

thats just the change in behavior / specs which is the problem. I don't see where the specs you linked are saying that, and that hasn't been the case in the past.

constraint values MAY be primitive types as possible in JSON, or represented as strings that are castable with the type and format rules of the field.
[…]
Constraints MUST be applied on the logical representation of field values and constraint values.

which does not rule out any which is the default type if non is specified.

Any type or format is accepted. When converting from physical to logical representation, the behaviour should be similar to String field type.

If the default datatype any no longer supports constraints, the specs should be updated to reflect that. But right now the specs say that contraints use logical representation, and tht any accepts any input when creating said logical representation (which acts like string)

[roll]

@duncdrum
I mean the spec says that e.g. min/maxLength constraints are not applicable to the type any:

Previously this kind of field (e.g. any with minLength constraints) would silently skip the constraint check (it's not possible to check for any as it's not typed data) but at some point it was proposed to catch these kind of problems on the schema level - frictionlessdata/tableschema-py#264