Workflow: Support user-defined diagnostic metrics and plots

[paul-buerkner]

The new BasicWorkflow objects are really cool! I also like the built-in (default) diagnostic metrics and plots that we can call via workflow.compute_diagnostics and workflow.plot_diagnostics, respectively.

It would be cool if you could allow users to specify and pass their own metrics or plot functions as long as they follow some fixed in and output structure that aligns with the built-in metrics and plots.

Example:

# computes all built-in default metrics
workflow.compute_diagnostics(test_data=300)

# also compute a custom metric:
def MAE ...
workflow.compute_diagnostics(test_data=300, custom_metrics = dict(MAE = MAE))

So custom_metrics could be a dictionary of functions. The keys of the dictionary would be used as names in the output data frame. If the functions don't follow the expected in- and output structure, an informative error is raised.

It would also be cool if one could choose which built-in metrics to compute. For example via a simple metrics argument taking a list.

# computes all built-in default metrics
workflow.compute_diagnostics(test_data=300, metrics = ["NRMSE"])