Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add evaluation type for cams283, tweak rtol fairmode #1329

Merged
merged 4 commits into from
Aug 30, 2024
Merged

add evaluation type for cams283, tweak rtol fairmode #1329

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
6c376f0
add evaluation type for cams283, tweak rtol fairmode
charlienegri Aug 29, 2024
cf129e1
add dummy test just to show that the long2 option doe not crash the code
charlienegri Aug 29, 2024
afb0a48
instead of having a new option, skip using make_period_ys if we have …
charlienegri Aug 30, 2024
6ddfcf8
cleanup unused type
charlienegri Aug 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion pyaerocom/aeroval/fairmode_stats.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ def fairmode_stats(obs_var: str, stats: dict) -> dict:
assert np.isclose(
rmsu * beta_mqi,
np.sqrt((bias) ** 2 + (mod_std - obs_std) ** 2 + (2 * obs_std * mod_std * (1 - R))),
rtol=1e-3,
rtol=1e-2,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is related to the rounding done for the statistical values.
dc40c69 reduced the accuracy from 1e-10 to 1e-6. Basically, the formula written there is not true for rounded values. I don't know why it is importand for fairmode.

I am not sure what the test here tries to achieve? It compares two different calculations for rms (beta_mqi = rms/rmsu), but the second one with rounded values?

), "failed MQI check"

fairmode = dict(
Expand Down
9 changes: 9 additions & 0 deletions pyaerocom/scripts/cams2_83/evaluation.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@

class EvalType(str, Enum):
LONG = "long"
LONG2 = "long2"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove LONG2

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks I had forgotten this

SEASON = "season"
WEEK = "week"
DAY = "day"
Expand All @@ -43,6 +44,14 @@ def freqs_config(self) -> dict:
main_freq="daily",
forecast_evaluation=True,
)

if self == "long2": # this is to have the same frequencies than 'long' but avoid L81
return dict(
freqs=["daily", "monthly"],
ts_type="hourly",
main_freq="daily",
forecast_evaluation=True,
)

if self == "season":
return dict(
Expand Down
10 changes: 10 additions & 0 deletions tests/cams2_83/test_cams2_83_cli_mos.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,16 @@ def test_eval_mos_dummy(
assert "no output available" in caplog.text


def test_eval_mos_dummy_2(
tmp_path: Path,
caplog,
):
options = f"long2 2024-03-01 2024-05-12 --data-path {tmp_path} --coldata-path {tmp_path} --name 'Test'"
result = runner.invoke(app, options.split())
assert result.exit_code == 0
assert "no output available" in caplog.text


@pytest.mark.usefixtures("fake_CAMS2_83_Processer", "reset_cachedir")
def test_eval_mos_standard(tmp_path: Path, coldata_mos: Path, caplog):
options = f"day 2024-03-01 2024-03-01 --data-path {tmp_path} --coldata-path {coldata_mos} --cache {tmp_path} --id mos-colocated-data --name 'Test'"
Expand Down