Pydantic v2 Overhaul [DNM yet] #321

Lnaden · 2023-08-09T14:21:27Z

Description

This PR upgrades the entire QCElemental package to use the Pydantic v2 system, entirely removing any and all calls to the v1 API.

This PR also preserves QCElemental's API as best as it can, throwing deprecation warnings in places where we may want to upgrade; which can be a point of discussion.

All tests pass locally, however, this should NOT be merged yet until a formal discussion can be had. Given the massive change this PR introduces, and considering how many packages depend on this repo, I want everyone to be sure about a full stack upgrade before this is merged. This is a medium priority, but no rush PR (@loriab and @bennybp specifically)

Happy to talk and explain any and everything.

Changelog description

Updates to Pydantic v2, rendering v1 incompatible.

Status

Code base linted
Ready to go

…testing so very much a WIP still.

…and only use json on numpy arrays for now.

qcelemental/models/common_models.py

qcelemental/models/types.py

…ires python 3.7.1+, but apparently pinning python to `^3.7` which is supposed to be equivalent to `>=3.7, <4.0` doesn't let the stack resolve, so pinning python to `^3.7.1` was the correct solution, even though it should have resolved...

codecov · 2023-08-10T16:08:54Z

Codecov Report

Merging #321 (75e6aee) into master (c9e308c) will increase coverage by 0.04%.
Report is 2 commits behind head on master.
The diff coverage is 82.91%.

Additional details and impacted files

…ther. I moved the commented block in case we needed to change it later.

…te, so split it up the hard way since there doesn't appear to be any other way to acess the name. This is the single most frustrating bug to track down since the error is thrown in pydantic but doesn't backtrace through the pydantic.__getattr__ -> `AutoPydanticDocGenerator` -> `__get__` -> `doc_formatter` -> `annotation.__name__` function because all the exceptions are caught the whole way up. GYA!

Lnaden · 2023-08-10T18:44:41Z

e32570b might be the most frustrating little bug i fixed ever because we and pydantic do such a good job of trapping bugs all the way up the stack, and the actual issue was Python less than 3.10.

loriab

This all looks very good to me. (Though basemodels.py is beyond my understanding.) I think I've picked up enough to migrate psi4's pydantic usage.

I'm not sure what a good plan for next steps is. Merge to a pydantic-next branch on the MolSSI repo while the rest of the stack builds out? conda packages off qcarchive?

loriab · 2023-08-09T15:24:58Z

.github/workflows/CI.yaml

@@ -13,7 +13,6 @@ jobs:
      fail-fast: true
      matrix:
        python-version: ["3.7", "3.9", "3.11"]
-        pydantic-version: ["1", "2"]


might want to leave this line in for now to help the CI lanes "Required" labels cope.

loriab · 2023-08-09T16:11:06Z

qcelemental/models/basemodels.py

@@ -65,18 +84,24 @@ def parse_raw(cls, data: Union[bytes, str], *, encoding: Optional[str] = None) -
                raise TypeError("Input is neither str nor bytes, please specify an encoding.")

        if encoding.endswith(("json", "javascript", "pickle")):
-            return super().parse_raw(data, content_type=encoding)
+            # return super().parse_raw(data, content_type=encoding)
+            return cls.model_validate_json(data)


should model_validate_json work on javascript or pickle? (not that I know who uses javscript or pickle with this.

loriab · 2023-08-10T17:41:59Z

qcelemental/util/autodocs.py

I switched over to autodoc-pydantic Spring 2022 (or thereabouts), so I should check if this is still in use at all. Native Sphinx on pydantic is awful, so DGAS had something better, but autodoc-pydantic is far better still.

…ping and pre Python3.9 in tandem with typing_extensions Annotation Source information: beartype/beartype#42 Annotated checks against instances of _GenericAlias to see if you can support Data[type_info], e.g. NDArray[int] (in Python types since 3.9, and kinda of as _GenricAlias before that.) Prior to 3.9, Numpy implemented their own version of _GenericAlias which isn't Python _GenericAlias, so the types are not considered "Generic" when Annotated[NDArray, Metadata][type_info] does its thing. So. This fix code does the heavy lifting to re-cast the NDArray type with _GenericAlias from python typing. I've tried to reuse as much data from NDArray as I possibly can and still use np.ndarray (which is not np.typing.NDArray) to still correctly type hint np.ndarrays.

Lnaden · 2023-08-11T15:07:14Z

I have just spend 2 whole days chasing down the two most subtle interactions ever: One was in the autodoc code (pre Python 3.10 issue with annotations), and one was with Annotated + Numpy.typing.NDArray in pre Python 3.9 and how Annotated does instance checking against Python's native type _GenericAlias objects but NumPy implemented their own. GYAH.

I'll be able to cycle back around to your comments now @loriab.

loriab · 2023-08-15T05:09:24Z

Thanks for all your work on this so far, @Lnaden. I've updated through psi4 with pydantic v2 API and hit a couple serialization bugs/changes that I hope you can either fix or advise on.

(1) The Datum class doesn't like to be json serialized. (This shows up in psi4 vibrational analysis -- test tu4.) In qcelemental/tests/test_datum.py, this can be recreated by switching out .json() for .dict(). The error is shown below.

def test_to_dict(dataset):
    listans = [i * 4 / 3 for i in range(4)]
    ans = {"label": "an array", "units": "cm^-1", "data": listans, "comment": "freqs", "numeric": True}

    #dicary = dataset["ndarray"].dict()  # fine
    #dicary = dataset["ndarray"].model_dump()  # fine but fails on compare_recursive. error below
    #dicary = dataset["ndarray"].json()  # error below
    dicary = dataset["ndarray"].model_dump_json()  # error below
    assert compare_recursive(ans, dicary, 9)

# .json() or .model_dump_json() error
#E       pydantic_core._pydantic_core.PydanticSerializationError: Error serializing to JSON: PydanticSerializationError: Unable to serialize unknown type: array([0.        , 1.33333333, 2.66666667, 4.        ])

# .model_dump() error
#E       AssertionError: assert False
#E        +  where False = compare_recursive({'comment': 'freqs', 'data': [0.0, 1.3333333333333333, 2.6666666666666665, 4.0], 'label': 'an array', 'numeric': True, ...}, {'comment': 'freqs', 'data': array([0.        , 1.33333333, 2.66666667, 4.        ]), 'doi': None, 'glossary': '', ...}, 9)

(2) optking is using qcel.util.serialization.json_dumps to make an AtomicResult json serializable. https://github.com/psi-rking/optking/blob/master/optking/compute_wrappers.py#L84-L85 If I switch it to ret = json.loads(ret.model_dump_json()), it's fine (that is, the optimization succeeds, but I didn't examine the intermediates). Should this line need to change? It'd be nice for optking (no pydantic dependence) to not need to branch based on qcel.

…lization wrappers together. Who knew? Also fixed an issue where the default `model_dump` wasn't calling the correct settings.

Lnaden · 2023-08-15T13:47:20Z

@loriab Thanks for testing the downstream things for me to find some of these bugs.

(1) That was indeed a bug, two separate ones that. I had updated the dict method but not the model_dump method for Datum which is not a derivative of ProtoModel, so the code paths were different depending on which you called. I also learned that you cannot chain together WrapSerializer objects the way I had them, so I combined them into one function which fixes your use cases there.

(2) Not sure on what exactly to do here because I don't understand your expected outputs. Can you tell me what the function was expecting type wise and what it gets out in this v2 version? Explicit outputs would be super helpful.

I'm almost certain I can fix this on my end, I just need more details. I know that one difference in Pydantic between v1 and v2 that might impact this is in JSON encoding. In v1, the .json methods took the model and cast it to a full string-ified dict with output type str. in v2, it casts its model to a JSON-compatible dict where all keys and values are str, but returns a dict overall. I don't know if that's related to what you need here?

Thanks again for testing downstream things, this is a huge help in finding subtle interactions like the one from your (1)

loriab · 2023-08-15T15:19:09Z

Thanks very much!

(1b) Still on the Datum part, now it gets through all the serializing all the real ndarrays, but it balks on the complex (freq for transition states). If you change dict -> json in test_complex_array, it gives E pydantic_core._pydantic_core.PydanticSerializationError: Error serializing to JSON: PydanticSerializationError: Unable to serialize unknown type: 1j

@@ -133,7 +141,7 @@ def test_complex_array():
         "numeric": True,
     }
 
-    dicary = datum1.dict()
+    dicary = datum1.json()
     assert compare_recursive(ans, dicary, 9)

(1c) This next one (still on Datum), I don't need, and I don't know if its a behavior change, I just thought I'd run it by you since you've got a feel for serialization. It doesn't look like one can recreate the model once serialized to json because arrays end up as strings.

(in test_datum.py)

def test_to_dict(dataset):
    listans = [i * 4 / 3 for i in range(4)]
    ans = {"label": "an array", "units": "cm^-1", "data": listans, "comment": "freqs", "numeric": True}

    jstr = dataset["ndarray"].model_dump_json()
    print(jstr)
    dicary = json.loads(jstr)
#    dicary = dataset["ndarray"].model_dump()
    print(dicary)
    model = qcel.Datum(**dicary)
    print(model.model_dump())
    print(ans)

    assert compare_recursive(ans, dicary, 9)

self = <[AttributeError("'Datum' object has no attribute '__pydantic_extra__'") raised in repr()] Datum object at 0x7fa0246782d0>, label = 'an array', units = 'cm^-1'
data = '[0.0, 1.3333333333333333, 2.6666666666666665, 4.0]'

    def __init__(self, label, units, data, *, comment=None, doi=None, glossary=None, numeric=True):
        kwargs = {"label": label, "units": units, "data": data, "numeric": numeric}
        if comment is not None:
            kwargs["comment"] = comment
        if doi is not None:
            kwargs["doi"] = doi
        if glossary is not None:
            kwargs["glossary"] = glossary
    
>       super().__init__(**kwargs)
E       pydantic_core._pydantic_core.ValidationError: 1 validation error for Datum
E       data
E         Value error, Datum data should be float, Decimal, or np.ndarray, not <class 'str'>. [type=value_error, input_value='[0.0, 1.3333333333333333....6666666666666665, 4.0]', input_type=str]
E           For further information visit https://errors.pydantic.dev/2.1/v/value_error

qcelemental/datum.py:90: ValidationError
-------------------------------------------------------------------------------------------- Captured stdout call ---------------------------------------------------------------------------------------------
{"numeric":true,"label":"an array","units":"cm^-1","data":"[0.0, 1.3333333333333333, 2.6666666666666665, 4.0]","comment":"freqs"}
{'numeric': True, 'label': 'an array', 'units': 'cm^-1', 'data': '[0.0, 1.3333333333333333, 2.6666666666666665, 4.0]', 'comment': 'freqs'}

Lnaden · 2023-08-15T17:43:30Z

@loriab

Should have resolved 1b and 1c now.

(1b) Since serialization doesnt chain anymore, I have to handle all cases in one function instead of letting the type detection handle it. Complex NumPy arrays now are cast to list, and then the complexs are also cast to list. So this should be handled.

(1c) The output of JSON strings is now correctly a list instead of a string-ified list. Technically speaking, your code wouldn't work on master anyways since Dataum doesn't take a list input, but the error you pointed out where the json version of data was a str(list(...)) should now be a list(...) correctly.

loriab · 2023-08-15T19:38:14Z

1b definitely resolved, thanks!

(1d) What's your view on numpy floats? I can easily solve this in my code by casting to float, but as-is, one can create a Datum with a numpy float but it won't json-ize.

>>> import qcelemental as qcel
>>> rT=0.5
>>> import numpy as np
>>> asdf = qcel.Datum("a np float", "J/K", np.sum(rT / np.expm1(rT) - np.log(1 - np.exp(-rT))))
>>> print(asdf)
----------------------------------------
            Datum a np float            
----------------------------------------
Data:     1.7034991708355878
Units:    [J/K]
doi:      None
Comment:  
Glossary: 
----------------------------------------
>>> type(asdf.data)
<class 'numpy.float64'>
>>> asdf.model_dump()
{'numeric': True, 'label': 'a np float', 'units': 'J/K', 'data': 1.7034991708355878}
>>> asdf.model_dump_json()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/psi/toolchainconda/envs/py311pyd2/lib/python3.11/site-packages/pydantic/main.py", line 345, in model_dump_json
    return self.__pydantic_serializer__.to_json(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.PydanticSerializationError: Error serializing to JSON: PydanticSerializationError: Unable to serialize unknown type: 1.7034991708355878
>>>

Lnaden · 2023-08-15T20:47:08Z

I think thats something I can fix on my end, it should be casting to float correctly, dunno why its not. Probably has to do with the "automatically inferred output type of the serialization function based on the type hint" and I'm not being generous enough.

Lnaden · 2023-08-17T17:19:56Z

@loriab

(1d) Fixed this by forcing the NumPy scalars to be cast to Python types. Wheeeeeeeeeeeeeee edge cases

This had nothing to do with the type hint return idea unfortunately.

qcelemental/datum.py

+        try:
+            # Cast NumPy scalar data types to native Python data type
+            v = v.item()
+        except (AttributeError, ValueError):


loriab · 2023-08-17T17:50:17Z

Confirmed (1d) fixed! thanks, @Lnaden

loriab · 2023-08-18T18:37:23Z

Nice strategy with the deprecation warning on .dict and .json, btw. The first time through the full psi4 test suite, I triggered 89k of them :-) . Now fixed except for some choice leftovers so we know it won't break for the user.

Lnaden · 2023-08-18T19:04:43Z

Thanks. I tried to preserve the API on our side as much as possible while keeping the deprecation warning. I can remove the warning on our side and just let people use both. There really shouldn't be any harm in it, so I can cut that if its obnoxious.

loriab · 2023-08-18T20:51:35Z

Thanks. I tried to preserve the API on our side as much as possible while keeping the deprecation warning. I can remove the warning on our side and just let people use both. There really shouldn't be any harm in it, so I can cut that if its obnoxious.

I think it's good to have the warning. It makes for a nice warning hunt :-)

I agree on allowing both for the foreseeable future. It would have been vexing to hunt all the dict()s down before facing all the real pydantic v2 api issues.

And they're not obnoxious -- pytest collects them very nicely.

tests/pytests/test_addons.py: 3 warnings
tests/pytests/test_detci_opdm.py: 1 warning
tests/pytests/test_standard_suite.py: 26288 warnings
tests/pytests/test_raises.py: 4 warnings
tests/pytests/test_mcmurchie_davidson.py: 1 warning
tests/pytests/test_restart.py: 5 warnings
tests/pytests/test_mints.py: 2 warnings
tests/pytests/test_misc.py: 2 warnings
tests/pytests/test_scf_options.py: 16 warnings
tests/pytests/test_addons_qcschema.py: 9 warnings
tests/pytests/test_qcfractal.py: 224 warnings
tests/pytests/test_molden_writer.py: 9 warnings
tests/pytests/test_nbody.py: 708 warnings
tests/pytests/test_mp2.py: 6 warnings
tests/pytests/test_dft_benchmarks.py: 2208 warnings
tests/pytests/test_mp2d.py: 34 warnings
tests/pytests/test_ccresponse.py: 2 warnings
tests/pytests/test_qcschema.py: 26 warnings
tests/pytests/test_composite.py: 26 warnings
tests/pytests/test_compositejk.py: 112 warnings
tests/pytests/test_nbody_multi_level_qcschema.py: 252 warnings
tests/pytests/test_option.py: 1 warning
tests/pytests/test_optking.py: 177 warnings
tests/pytests/test_psi4.py: 30 warnings
tests/pytests/test_psi4_qcschema.py: 34 warnings
tests/pytests/test_pyside_cubegen.py: 1 warning
tests/pytests/test_dft_blocking.py: 10 warnings
tests/pytests/test_dipoles.py: 60 warnings
tests/pytests/test_erisieve.py: 11 warnings
tests/pytests/test_fchk_writer.py: 12 warnings
tests/pytests/test_fcidump_energy.py: 2 warnings
tests/pytests/test_geometric.py: 31 warnings
tests/pytests/test_gradients.py: 403 warnings
tests/pytests/test_tdscf_excitations.py: 70 warnings
tests/pytests/test_vibanalysis.py: 10374 warnings
tests/pytests/test_tdscf_products.py: 10 warnings
tests/pytests/test_weird_basis.py: 12 warnings
tests/pytests/test_wfn.py: 1 warning
  /psi/gits/QCElemental/qcelemental/models/basemodels.py:133: DeprecationWarning: The `dict` method is deprecated; use `model_dump` instead.
    warnings.warn("The `dict` method is deprecated; use `model_dump` instead.", DeprecationWarning)

…d as a function first before calling tolist

qcelemental/util/serialization.py

…ich calls different logic overall than to_jsonable_python for some reason I don't understand

qcelemental/models/common_models.py

-        from pydantic.v1.typing import ReprArgs
-    except ImportError:  # Will also trap ModuleNotFoundError
-        from pydantic.typing import ReprArgs
+    ReprArgs = Sequence[Tuple[Optional[str], Any]]


awvwgk · 2024-03-13T08:18:14Z

Is this effort still active? Having QCElemental with Pydantic v2 support would be great, however a smooth transition from v1 to v2 API would be appreciated. Currently QCElemental is relying on the v1 API provided in Pydantic v2, instead of directly switching to the v2 API and removing the support for v1, would it be possible to have both available dependent on which Pydantic API is available? An example for this could be the approach taken in the beanie package.

loriab · 2024-03-15T04:49:24Z

Is this effort still active? ...

It's still active, and it works (except for one line with optking). And downstream v2-proper PRs are largely ready for QCEngine and Psi4. It's frustrating that pydantic hasn't provided a per-model upgrade scheme because as is, whole software stacks need to be migrated together. Last I heard, OpenFF was having trouble with their v2 migration. Any updates, @mattwthompson ?

Thanks for the beanie model. I think that'd work pretty cleanly for QCEngine and maybe Psi4. QCElemental might need some completely separate files for v1/v2 since it access low-level pydantic stuff more. But I agree something needs to be done -- I'm translating new v2 models back to v1 just so I can actually use them in the stack.

mattwthompson · 2024-03-15T13:53:21Z

Our "proper" v2 migration is tied up in a huge mess of things and stalled - the crux of the thing being getting our custom pint registry to work in Pydantic models without breaking all of our downstream users' APIs

https://github.com/openforcefield/openff-models/blob/main/openff/models/types.py#L20-L37

loriab · 2024-03-16T03:45:30Z

Thanks for the update, @mattwthompson . After I get through these next papers I'll try a dual v1/v2 version of qcel. But I bet one wants a triple version for complete compatibility -- v1, v2 as v1, v2.

mattwthompson · 2024-04-02T19:46:23Z

qcelemental/models/basemodels.py

+# Encoders, to be deprecated at some point
+ndarray_encoder = {np.ndarray: lambda v: v.flatten().tolist()}


Is this used in these models? I see similar code elsewhere (https://github.com/MolSSI/QCElemental/pull/321/files#diff-451348f65352b6379290ba87c966a55901f8fd840a77da1f09bcaf8fbdcc7485L46-L49) but not using this variable

That might be leftover from when I was doing the initial refactor pass. They changed the way they do encoders for custom stuff to such a degree that I might have added that, then had to change the internals later on and never cycled back to this line.

Basically see the entire "types" file where I had to add a bunch of new validation logic: https://github.com/MolSSI/QCElemental/pull/321/files#diff-47aa66e9bfa663f2f9b398bd05273dad682d226378e90ba94d633f0bbf0a7885R1-R96

Lnaden added 6 commits August 1, 2023 16:29

Start migrating to Pydantic v2 still very much a WIP

4777b4b

Finish initial pass converting to v2 pydantic API. Have not done any …

538a62f

…testing so very much a WIP still.

More headway, starting on tests. have to be careful with serializers …

5bd215b

…and only use json on numpy arrays for now.

Why does nested serialization behavior change between versions? WHY!?

76b565b

Finally have all tests passing. Huzzah! Still need to cleanup

248676d

Handled CI and Lint. All tests pass locally.

a2e3501

Lnaden requested review from loriab and bennybp August 9, 2023 14:21

github-advanced-security bot found potential problems Aug 9, 2023

View reviewed changes

qcelemental/models/common_models.py Fixed Show fixed Hide fixed

qcelemental/models/types.py Fixed Show fixed Hide fixed

Lnaden added 2 commits August 10, 2023 12:05

fix black

e7f4549

Lnaden added 3 commits August 10, 2023 12:09

and apparently isort

4b4c977

So apparently I made a fun problem where black and isort undid each o…

541beb0

…ther. I moved the commented block in case we needed to change it later.

loriab reviewed Aug 10, 2023

View reviewed changes

Lnaden added 2 commits August 11, 2023 10:48

One last doc bug and black

ad7d43b

Fixed a serialization problem in Datum because you cannot chain seria…

a4d0ed2

…lization wrappers together. Who knew? Also fixed an issue where the default `model_dump` wasn't calling the correct settings.

Lnaden added 3 commits August 15, 2023 13:34

Fixed serialization of complex ndarrays in Datum

37fcc09

Dont cast listified complex or ndarray to string on jsonify of data

31f0b7c

black

ad3b633

Fixed NumPy native types being unknown on how to serialize by Pydantic

6af962b

github-advanced-security bot found potential problems Aug 17, 2023

View reviewed changes

qcelemental/datum.py

try:

# Cast NumPy scalar data types to native Python data type

v = v.item()

except (AttributeError, ValueError):

Check notice

Code scanning / CodeQL

Empty except

'except' clause does nothing but pass and there is no explanatory comment.

Lnaden added 2 commits August 30, 2023 11:28

Debugging more of the JSON serializing

01f19ae

Fixed typo in Array serialization to JSON where flatten was not calle…

f2b5989

…d as a function first before calling tolist

loriab mentioned this pull request Aug 30, 2023

PSA: Pydantic v1/v2 and the QCArchive + Psi4 stack #323

Open

Removed leftover debugging lines

2ca4b14

github-advanced-security bot found potential problems Aug 30, 2023

View reviewed changes

qcelemental/util/serialization.py Fixed Show fixed Hide fixed

For serializing models, just try dumping them with model_dump_json wh…

75e6aee

…ich calls different logic overall than to_jsonable_python for some reason I don't understand

github-advanced-security bot found potential problems Aug 30, 2023

View reviewed changes

mattwthompson reviewed Apr 2, 2024

View reviewed changes

This was referenced Sep 10, 2024

Csse pyd2 510 #347

Merged

Csse pyd2 510 Part 2 #348

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pydantic v2 Overhaul [DNM yet] #321

Pydantic v2 Overhaul [DNM yet] #321

Lnaden commented Aug 9, 2023

codecov bot commented Aug 10, 2023 •

edited

Loading

Lnaden commented Aug 10, 2023

loriab left a comment

loriab Aug 9, 2023

loriab Aug 9, 2023

loriab Aug 10, 2023

Lnaden commented Aug 11, 2023

loriab commented Aug 15, 2023

Lnaden commented Aug 15, 2023

loriab commented Aug 15, 2023

Lnaden commented Aug 15, 2023

loriab commented Aug 15, 2023

Lnaden commented Aug 15, 2023

Lnaden commented Aug 17, 2023 •

edited

Loading

loriab commented Aug 17, 2023

loriab commented Aug 18, 2023

Lnaden commented Aug 18, 2023

loriab commented Aug 18, 2023

awvwgk commented Mar 13, 2024

loriab commented Mar 15, 2024

mattwthompson commented Mar 15, 2024

loriab commented Mar 16, 2024

mattwthompson Apr 2, 2024

Lnaden Apr 2, 2024

Lnaden Apr 2, 2024

		# Encoders, to be deprecated at some point
		ndarray_encoder = {np.ndarray: lambda v: v.flatten().tolist()}

Pydantic v2 Overhaul [DNM yet] #321

Are you sure you want to change the base?

Pydantic v2 Overhaul [DNM yet] #321

Conversation

Lnaden commented Aug 9, 2023

Description

Changelog description

Status

codecov bot commented Aug 10, 2023 • edited Loading

Codecov Report

Lnaden commented Aug 10, 2023

loriab left a comment

Choose a reason for hiding this comment

loriab Aug 9, 2023

Choose a reason for hiding this comment

loriab Aug 9, 2023

Choose a reason for hiding this comment

loriab Aug 10, 2023

Choose a reason for hiding this comment

Lnaden commented Aug 11, 2023

loriab commented Aug 15, 2023

Lnaden commented Aug 15, 2023

loriab commented Aug 15, 2023

Lnaden commented Aug 15, 2023

loriab commented Aug 15, 2023

Lnaden commented Aug 15, 2023

Lnaden commented Aug 17, 2023 • edited Loading

loriab commented Aug 17, 2023

loriab commented Aug 18, 2023

Lnaden commented Aug 18, 2023

loriab commented Aug 18, 2023

awvwgk commented Mar 13, 2024

loriab commented Mar 15, 2024

mattwthompson commented Mar 15, 2024

loriab commented Mar 16, 2024

mattwthompson Apr 2, 2024

Choose a reason for hiding this comment

Lnaden Apr 2, 2024

Choose a reason for hiding this comment

Lnaden Apr 2, 2024

Choose a reason for hiding this comment

codecov bot commented Aug 10, 2023 •

edited

Loading

Lnaden commented Aug 17, 2023 •

edited

Loading