Reset descriptor properties to default value and keep it out of metadata export

[fjuniorr]

What is the proper way to "reset" a property in a descriptor so that it don't get exported in calls to to_descriptor() and similars (eg. to_json())?

For example, imagine that after cleaning the data for the field:

name: memo
type: string
missingValues: 
  - 'NA'

The missingValues property no longer applies. How can I get the following programmatically

name: memo
type: string

I've tried

from frictionless import Resource, settings

url = 'https://raw.githubusercontent.com/splor-mg/datapackage-reprex/main/reprex/20230711T100325/resource.yaml'

resource = Resource(url)

resource.schema.update_field(name='memo', descriptor={'missingValues': settings.DEFAULT_MISSING_VALUES.copy()})
print(resource.schema.get_field('memo').to_yaml())

But I still get the default value in the metadata export:

name: memo
type: string
missingValues:
  - ''

[roll]

@fjuniorr
Thanks it's a good catch

@shashigharti
Can you please take a look? Shall we provide a way to unset props in this step?

[shashigharti]

Hi @fjuniorr,

Temporary solution could be:

url = 'https://raw.githubusercontent.com/splor-mg/datapackage-reprex/main/reprex/20230711T100325/resource.yaml'

resource = Resource(url)
class custom_step(Step):
    def __init__(self, *, fields_to_remove=None):
        self.fields_to_remove = fields_to_remove

    def transform_resource(self, resource):
        schema_descriptor =  resource.schema.to_descriptor()
        if self.fields_to_remove:
            for property in self.fields_to_remove:
                schema_descriptor = self.remove_field(schema_descriptor, property) 
        resource.schema = Schema.from_descriptor(schema_descriptor)

    def remove_field(self, descriptor, field_to_remove):
        if isinstance(descriptor, dict):
            for key in list(descriptor.keys()):
                if key == field_to_remove:
                    del descriptor[key]
                else:
                    self.remove_field(descriptor[key], field_to_remove)
        elif isinstance(descriptor, list):
            for item in descriptor:
                self.remove_field(item, field_to_remove)
        return descriptor
        
pipeline = Pipeline(steps=[custom_step(fields_to_remove = ['missingValues'])])
target = resource.transform(pipeline)
print(resource.schema.get_field('memo').to_yaml())

@roll I think it it would be better to provide option to unset props like you mentioned.

[fjuniorr]

Thanks @shashigharti!