Pyff failing to load metadata file and publishing incomplete metadata

[salaun-urennes1]

Hello,

We've had an incident in our SAML metadata processing using pyff. Our federation registry generates intermediate metadata that are then processed with pyff to aggregate eduGAIN downstream metadata. Pyff was unable to parse our intermediate metadata file but processing did not raise a fatal error and published incomplete metadata.

Attached are two simple metadata files and a pyff configuration files that reveal the problem in a simplified context. What prevents loading of metadata-bad.xml is the empty "saml:AttributeValue/saml:AttributeValue". Schema validation of metadata-bad.xml succeeds with xmlsectool.

We are using latest version of git://github.com/leifj/pyFF.git

Here is the output we get from pyf; note the 0 status code:

/opt/pyff-latest/bin/pyff --loglevel=INFO bug300316.fd ;echo "Status: $?"

ERROR:root:fetching 'file://metadata-bad.xml' generated an exception: 'NoneType' object has no attribute 'strip'

total size: 1
selected: 1
idps: 0

sps: 1

Status: 0

I added traces in the code and found out the error occurs in the _index_entity() method of the mdrepo.py file. I'm not familiar enough with python and pyff code to suggest fixes in the code.

renater-sample.zip

[leifj]

I can't find that method anywhere in the master branch. Can you check the name?

[leifj]

could it be in with_entity_attributes you're seeing the error?

[salaun-urennes1]

Sorry for the wrong reference ; the investigations I did were on revision 05d6e0283e699afda411143a5826436188916c1c from 2014.

Back in the revision I noticed a call to mdrepo/import_metadata() and then to self._index_entity that failed.

Do you need me to find the equivalents in the latest revision of the code?
I noticed that you clsed the issue; does it mean it should be fixed in master?

[salaun-urennes1]

I just did a git pull and tried to reproduce the problem: the fix works fine.
Thanks for providing the fixed version so quickly.

I am also concerned with pyff's behavior when a metadata source could not be loaded, as previously reported in #62. Should I reopen this issue?

[leifj]

#62 is still open isn't it?

I think we need to have a good solution for "stop unless everything is ok" too but I don't want this to be the default because in many cases, downstream consumers can cope with minor outages.