Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Define common-metadata operations on split attribute dictionaries. #77

Draft
wants to merge 13 commits into
base: splitattrs_ncsave_redo
Choose a base branch
from

Conversation

pp-mo
Copy link
Owner

@pp-mo pp-mo commented Aug 14, 2023

WIP
just testing
provisional changes to common-metadata to accommodate split attribute dictionaries

No new function yet -- just to see if it breaks any existing tests
Will need additional tests for how this actually functions with split-attr testcases

EDIT: this actually works now, and should be good, subject to first merging SciTools#5410 and rebasing/retargetting this as required.

@codecov-commenter
Copy link

codecov-commenter commented Aug 14, 2023

Codecov Report

❗ No coverage uploaded for pull request base (splitattrs_ncsave_redo@cbd7167). Click here to learn what that means.
The diff coverage is n/a.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

@@                    Coverage Diff                    @@
##             splitattrs_ncsave_redo      #77   +/-   ##
=========================================================
  Coverage                          ?   89.44%           
=========================================================
  Files                             ?       89           
  Lines                             ?    22590           
  Branches                          ?     5422           
=========================================================
  Hits                              ?    20205           
  Misses                            ?     1639           
  Partials                          ?      746           

@pp-mo
Copy link
Owner Author

pp-mo commented Aug 14, 2023

Note:

legacy tests do pass ! 👍
remaining:

  • tests
  • decide whether the lenient cases should have some more split-specific behaviour, and implement if needed

@pp-mo pp-mo changed the title Define common-metadata operartions on split attribute dictionaries. Define common-metadata operations on split attribute dictionaries. Aug 14, 2023
@pp-mo pp-mo force-pushed the splitattrs_ncsave_redo_commonmeta branch from 255eabb to 76a3974 Compare August 23, 2023 17:42
@pp-mo
Copy link
Owner Author

pp-mo commented Aug 23, 2023

Hmm, we are not done with this, as seen in doc-tests failures.
I think because we need to allow for operations between different metadata classes ?
I had thought that the common metadata code converted inputs to same type to avoid that -- need to look again.

@pp-mo
Copy link
Owner Author

pp-mo commented Aug 24, 2023

Hmm, we are not done with this, as seen in doc-tests failures. I think because we need to allow for operations between different metadata classes ? I had thought that the common metadata code converted inputs to same type to avoid that -- need to look again.

Ok I think the latest commit fixes it.

The logic here is that different Metadata types may not be combined in operations.
But that doesn't mean that all CubeMetadata.attributes must be a CubeAttrsDict. Naturally not, since the .difference operation produces a CubeMetdata whose .attributes is a pair(tuple).
So, the content of the "attributes" property in a Cube is controlled, but CubeMetadata objects themselves can be more various.

By "promoting" left+right to split-dicts where required, I think this gives the correct behaviour.
There are no specific tests for this usage (but the usage here in the docs is putting a regular dict into a CubeMetadata -- this is the doctest that failed).
However, I don't think we need to add any, since this is rather a corner-case : because actual Cube.attributes are always constrained to be a CubeAttrsDict.

trexfeathers and others added 6 commits August 24, 2023 11:34
* Clarification in CubeAttrsDict examples.

* CubeAttrsDict fix docstring typo.

* Raise awareness of split attributes in user guide.

* What's New entry.

* Changes to metadata documentation.
* Add docs and future switch, no function yet.

* Typing enables code completion for Cube.attributes.

* Make roundtrip checking more precise + improve some tests accordingly (cf. SciTools#5403).

* Rework all tests to use common setup + results-checking code.

* Saver supports split-attributes saving (no tests yet).

* Tiny docs fix.

* Explain test routines better.

* Fix init of FUTURE object.

* Remove spurious re-test of FUTURE.save_split_attrs.

* Don't create Cube attrs of 'None' (n.b. but no effect as currently used).

* Remove/repair refs to obsolete routines.

* Check all warnings from save operations.

* Remove TestSave test numbers.

* More save cases: no match with missing, and different cube attribute types.

* Run save/roundtrip tests both with+without split saves.

* Fix.

* Review changes.

* Fix changed warning messages.

* Move warnings checking from 'run' to 'check' phase.

* Simplify and improve warnings checking code.

* Fix wrong testcase.

* Minor review changes.

* Fix reverted code.

* Use sets to simplify demoted-attributes code.

* WIP

* Working with iris 3.6.1, no errors TestSave or TestRoundtrip.

* Interim save (incomplete?).

* Different results form for split tests; working for roundtrip.

* Check that all param lists are sorted.

* Check matrix result-files compatibility; add test_save_matrix.

* test_load_matrix added; two types of load result.

* Finalise special-case attributes.

* Small docs tweaks.

* Add some more testcases,

* Ensure valid sort-order for globals of possibly different types.

* Initialise matrix results with legacy values from v3.6.1 -- all matching.

* Add full current matrix results, i.e. snapshot current behaviours.

* Review changes : rename some matrix testcases, for clarity.
@pp-mo pp-mo force-pushed the splitattrs_ncsave_redo_commonmeta branch from d06f589 to 9d17da4 Compare October 10, 2023 12:23
@pp-mo pp-mo force-pushed the splitattrs_ncsave_redo_commonmeta branch from dae3a28 to b00b37f Compare November 15, 2023 13:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants