Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

em_framework: Improve log messages, warning and errors #300

Merged
merged 3 commits into from
Nov 17, 2023

Conversation

EwoutH
Copy link
Collaborator

@EwoutH EwoutH commented Oct 29, 2023

Improve the log messages, warning and errors in the em_framework, by:

  • Making them more explicit
  • Including what the variable value or type not expected currently is
  • Including what the variable value or type not expected should be
  • Stating the explicit type of warning or error (DeprecationWarning, ValueError, etc.)
  • Converting them to F-strings (for better in-code readability)
  • Formatting warnings and errors with a Capital letter, while leaving log-statements lowercase.

This will make debugging easier, allowing more time for model development and experimentation.

There are two placed where it's mentioned that a function is deprecated, but not when they will be removed. I added TODO's there, I think we should decide on a version in which we want to remove them.

@coveralls
Copy link

Coverage Status

coverage: 80.893%. remained the same when pulling 42477c9 on improve-warnings-and-errors into b76b487 on master.

@EwoutH EwoutH changed the title Improve log messages, warning and errors Improve log messages, warning and errors in em_framework Oct 29, 2023
@quaquel
Copy link
Owner

quaquel commented Oct 29, 2023

This looks very good and very useful. I'll try to take a look over the coming days.

@@ -346,14 +345,16 @@ def __init__(self, msis, n_processes=None, maxtasksperchild=None, **kwargs):
if isinstance(n_processes, int):
if n_processes > 0:
if max_processes < n_processes:
warnings.warn(f"the number of processes cannot be more then {max_processes}")
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why did you drop userwarning here?

ema_workbench/em_framework/evaluators.py Outdated Show resolved Hide resolved
# TODO: We should determine a version in which this method is removed and add that to the deprecation warning
warnings.warn(
"The 'retrieve_output' method is deprecated and will be removed in a future version. Use 'model.output' instead.",
DeprecationWarning,
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This has been deprecated for several years. We might consider removing it.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added that it will be removed in version 3.0

warnings.warn(
"HyperVolume is deprecated, use ArchiveLogger and HypervolumeMetric instead",
warnings.DeprecationWarning,
DeprecationWarning,
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was deprecated in the latest version, so it would make sense to remove it either in the next version or the one thereafer.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added that it will be removed in version 3.0

raise EMAError(f"outcome {self.name} should be a collection")
raise EMAError(
f"Outcome {self.name} should be a collection, but is a {type(values)}: {values}"
)
return values
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this exception was recently triggered in some work of one of my PhD students. so this error is already better. Not sure about printing the full values because this could be something quite big.

Also, this might be better as a TypeError?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the idea behind EMAError? Are there different types of EMAErrors? Would it be a user mistake, system failure, or can it be both?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EMEError is a catchall error for stuff within the workbench going wrong. The main use case is if something goes wrong in the interaction between the workbench and the model we are connecting with. This can be a user mistake but also some system failure.

There is also the CaseError, which is exclusively used if a single experiment fails (e.g., a numerical error in a vensim model).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed printing the full values and changed to TypeError.

@EwoutH
Copy link
Collaborator Author

EwoutH commented Oct 29, 2023

Thanks for reviewing so quickly. I will work on you comments after #299 is finished.

Improve the log messages, warning and errors, by:
- Making them more explicit
- Including what the variable value or type not expected currently is
- Including what the variable value or type not expected should be
- Stating the explicit type of warning or error (DeprecationWarning, ValueError, etc.)
- Converting them to F-strings (for better in-code readability)
- Formatting warnings and errors with a Capital letter, while leaving log-statements lowercase.

This will make debugging easier, allowing more time for model development and experimentation.

There are two placed where it's mentioned that a function is deprecated, but not when they will be removed. I added TODO's there, I think we should decide on a version in which we want to remove them.
@EwoutH
Copy link
Collaborator Author

EwoutH commented Nov 16, 2023

Processed the feedback. Added that both deprecated functions will be removed in 3.0.

Squash and merge when ready.

@EwoutH EwoutH changed the title Improve log messages, warning and errors in em_framework em_framework: Improve log messages, warning and errors in em_framework Nov 16, 2023
@EwoutH EwoutH changed the title em_framework: Improve log messages, warning and errors in em_framework Improve log messages, warning and errors in em_framework Nov 16, 2023
@EwoutH EwoutH changed the title Improve log messages, warning and errors in em_framework em_framework: Improve log messages, warning and errors Nov 16, 2023
@EwoutH EwoutH merged commit 2cfca39 into master Nov 17, 2023
20 checks passed
@EwoutH EwoutH deleted the improve-warnings-and-errors branch November 17, 2023 09:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants