[inbox] Inbox of bonus features

[rufuspollock]

• Special modes for ckan => f11s conversion b/c default mode is very unmagical. e.g.
  • Expansive mode: add title from name on resource, ...
  • Strict mode: do work to make sure it is a valid f11s object e.g. name is lower-cased on resource and package, add resources empty array if no array, add name
• Map a CKAN data dictionary to table schema

Strict mode

Name and title properties

We have existing tests in test_converter.py that ensure "name" and "title" are processed stricly:

    def test_resource_name_is_used_if_theres_no_title(self):
        resource = {
            'name': 'gdp',
            'title': None,
        }
        self.datapackage.resources[0].descriptor.update(resource)
        result = converter.datapackage_to_dataset(self.datapackage)
        resource = result.get('resources')[0]
        self.assertEquals(result.get('resources')[0].get('name'),
                          resource['name'])

    def test_resource_title_is_used_as_name(self):
        resource = {
            'name': 'gdp',
            'title': 'Gross domestic product',
        }

        self.datapackage.resources[0].descriptor.update(resource)
        result = converter.datapackage_to_dataset(self.datapackage)
        self.assertEquals(result.get('resources')[0].get('name'),
                          resource['title'])

Those have been refactored like this for now:

    def test_name_is_used_if_theres_no_title(self):
        indict = {'name': 'gdp'}
        out = converter.resource(indict)
        assert out.get('name') == indict['name']

    def test_resource_title_is_used_as_name(self):
        indict = {
            'name': 'gdp',
            'title': 'Gross domestic product',
        }

        out = converter.resource(indict)
        assert out.get('name') == indict['title']

Sluggifying tags

In strict mode, we may have to sluggify CKAN tags when converting keywords from Frictionless to tags in CKAN. Something like:

outdict['tags'] = [
    {'name': slugify.slugify(keyword).lower()}
    for keyword in outdict['keywords']
]

Map a CKAN data dictionary to table schema

• Add a function to do this mapping.
  • Data dictionary is what you get from the datastore - https://docs.ckan.org/en/2.8/maintaining/datastore.html#data-dictionary
  • https://docs.ckan.org/en/2.8/maintaining/datastore.html#fields