Retesting Strategy

[jbusecke]

Since our tests keep evolving we need a way to regularly retest datasets and ensure consistency.

• Moving stores from QC to non-qc catalog in case new tests discover broken stores that have previously slipped passed our guards (I am mostly worriying about incomplete stores resulting from partial url lists like in Incomplete file listings jbusecke/pangeo-forge-esgf#46 and Datasets whose temporal coverage differs between nodes are not fully ingested #53).
  • This case is related to How to overwrite. We need a way to reliably trigger a new write of a given iid #76, and we need to see how, when we want to try to overwrite all?/parts of the non-qc datasets to see if the newest recipe version produces a passing store.
• Moving stores from non-QC to the QC store. This is mainly for erroneously failed datasets (e.g. All 'areacello' data failed tests.  #146)