DM-47325: Add API for parsing butler dataset URIs (butler and ivo)

[timj]

Checklist

• ran Jenkins
• added a release note for user-visible changes to doc/changes
• (if changing dimensions.yaml) make a copy of dimensions.yaml in configs/old_dimensions

[codecov]

Codecov Report

Attention: Patch coverage is 98.90110% with 1 line in your changes missing coverage. Please review.

Project coverage is 89.46%. Comparing base (d632886) to head (2cbb8f0).
Report is 11 commits behind head on main.

✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
python/lsst/daf/butler/_labeled_butler_factory.py	87.50%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1113      +/-   ##
==========================================
+ Coverage   89.45%   89.46%   +0.01%     
==========================================
  Files         366      366              
  Lines       48684    48773      +89     
  Branches     5897     5907      +10     
==========================================
+ Hits        43548    43636      +88     
  Misses       3721     3721              
- Partials     1415     1416       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[gpdf]

Responded to request for feedback on ivo: URIs.

[timj]

@rra does this approach work for you in the cutout service?

factory = butler_factory.bind(access_token=token)
...
ref = Butler.get_dataset_from_uri(dataset_uri, factory=factory)

?

[rra]

The cutout service passes a Butler instance into the backend, so the pattern looks like:

butler_factory = LabeledButlerFactory()

def _get_backend(label: str, token: str) -> ImageCutoutBackend:
    # Called for each cutout
    factory = butler_factory.bind(access_token=token)
    butler = factory.create_butler(label=label)
    # ...
    return ImageCutoutBackend(butler, projection_finder, output, tmpdir)

Is that what you had in mind? Basically move the access token parameter to create_butler to an intermediate step to create a factory with a bound token? If so, that would be fine here.

[timj]

Ah. No. It's probably the wrong API for you. Somewhere you are parsing the butler:// URI and I'm trying to provide code here that will hide the URI structure from you (so that we can also support the new ivo:// URIs). This PR creates two APIs: one that parses the URI and returns the butler repo label and the UUID, and another API (that is the one I talk about here) that lets you retrieve the DatasetRef directly from the URI and a butler factory. Maybe get_dataset_from_uri should return the Butler instance along with the DatasetRef?

[rra]

Maybe I could pass the return value of bind into the backend along with the URIs as-is without parsing them and then the backend can do whatever it needs to do? That would be even more convenient. In other words, have the constructor of ImageCutoutBackend take a Butler factor instead of a Butler instance.

[timj]

I think passing the bound factory and URI into whatever is wanting to know the DatasetRef is what we want here.

[stvoutsin]

Overall looks good to me, I think the new methods make sense and are useful. Only a couple thoughts/suggestions here

[stvoutsin]

I'm not sure what is expected of query param keys in terms of case sensitivity. But I think no harm in treating those as case-insensitive as well?

[timj]

In theory we are generating these IVO IDs so I don't really want the added complication of converting the dict to a dict with case insensitive keys (I have a recollection of using such a special dict at some point but I'm not sure where it was).

[gpdf]

I'm fine with this now, though we are still discussion what will go into the authority field in the IVOID versus what will go in the resource key field (hostname and path fields, colloquially speaking).

I am assured that that's not going to require additional work on the code affected by this PR.