DM-47980: Add an option to include dimension records into general query result

[andy-slac]

The GeneralQueryResults.iter_tuples method returned DataIds without dimension records. In some cases (e.g. for obscore export) it would be very useful to include records in the same result to avoid querying them separately. New method with_dimension_records is added to the class to trigger adding fields from all dimension records into returned page. This will produce many duplicates for some dimensions (e.g. instrument) but it keeps page structure simple.

This adds one attribute to the GeneralResultSpec class, will need some care with Butler server compatibility.

Checklist

• ran Jenkins
• added a release note for user-visible changes to doc/changes
• (if changing dimensions.yaml) make a copy of dimensions.yaml in configs/old_dimensions

[codecov]

Codecov Report

Attention: Patch coverage is 88.09524% with 15 lines in your changes missing coverage. Please review.

Project coverage is 89.50%. Comparing base (d1c5477) to head (df7a350).
Report is 3 commits behind head on main.

✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
.../lsst/daf/butler/queries/_general_query_results.py	83.05%	8 Missing and 2 partials ⚠️
...tler/direct_query_driver/_result_page_converter.py	88.46%	1 Missing and 2 partials ⚠️
...hon/lsst/daf/butler/remote_butler/_query_driver.py	83.33%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1135      +/-   ##
==========================================
- Coverage   89.51%   89.50%   -0.01%     
==========================================
  Files         366      366              
  Lines       48894    48998     +104     
  Branches     5916     5943      +27     
==========================================
+ Hits        43766    43855      +89     
- Misses       3719     3729      +10     
- Partials     1409     1414       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[TallJimbo]

I think you've made this more difficult and less memory-efficient than it needed to be by making the cached/skypix columns appear like regular result columns (a couple of line comments on that subject), but it's possible I just didn't see the motivation. Looks good aside from that question.