diff --git a/docs/contributing.md b/docs/contributing.md index 291e6db4..69e4dc88 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -30,14 +30,14 @@ A new score or metric should be developed on a separate feature branch, rebased - The implementation of the new metric or score in xarray, ideally with support for pandas and dask - 100% unit test coverage - A tutorial notebook showcasing the use of that metric or score, ideally based on the standard sample data - - API documentation (docstrings) which clearly explain the use of the metrics + - API documentation (docstrings) using [Napoleon (google)](https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_google.html) style, making sure to clearly explain the use of the metrics - A reference to the paper which described the metrics, added to the API documentation - For metrics which do not have a paper reference, an online source or reference should be provided - For metrics which are still under development or which have not yet had an academic publication, they will be placed in a holding area within the API until the method has been properly published and peer reviewed (i.e. `scores.emerging`). The 'emerging' area of the API is subject to rapid change, still of sufficient community interest to include, similar to a 'preprint' of a score or metric. - All merge requests should comply with the coding standards outlined in this document. Merge requests will undergo both a code review and a science review. The code review will focus on coding style, performance and test coverage. The science review will focus on the mathematical correctness of the implementation and the suitability of the method for inclusion within 'scores'. +All merge requests should comply with the coding standards outlined in this document. Merge requests will undergo both a code review and a science review. The code review will focus on coding style, performance and test coverage. The science review will focus on the mathematical correctness of the implementation and the suitability of the method for inclusion within 'scores'. - A github ticket should be created explaining the metric which is being implemented and why it is useful. +A github ticket should be created explaining the metric which is being implemented and why it is useful. ### Development Process for a Correction or Improvement diff --git a/src/scores/continuous.py b/src/scores/continuous.py index c88f8367..fc870656 100644 --- a/src/scores/continuous.py +++ b/src/scores/continuous.py @@ -6,31 +6,38 @@ def mse(fcst, obs, reduce_dims=None, preserve_dims=None, weights=None): - """ + """Calculates the mean squared error from forecast and observed data. - Returns: - - By default an xarray containing a single floating point number representing the mean absolute - error for the supplied data. All dimensions will be reduced. - - Otherwise: Returns an xarray representing the mean squared error, reduced along - the relevant dimensions and weighted appropriately. + Dimensional reduction is not supported for pandas and the user should + convert their data to xarray to formulate the call to the metric. At + most one of reduce_dims and preserve_dims may be specified. + Specifying both will result in an exception. Args: - - fcst: Forecast or predicted variables in xarray or pandas - - obs: Observed variables in xarray or pandas - - reduce_dims: Optionally specify which dimensions to reduce when calculating MSE. - All other dimensions will be preserved. - - preserve_dims: Optionally specify which dimensions to preserve when calculating MSE. All other - dimensions will be reduced. As a special case, 'all' will allow all dimensions to - be preserved. In this case, the result will be in the same shape/dimensionality as - the forecast, and the errors will be the squared error at each point (i.e. single-value - comparison against observed), and the forecast and observed dimensions must match - precisely. - - weights: Not yet implemented. Allow weighted averaging (e.g. by area, by latitude, by population, custom) - - Notes: - - Dimensional reduction is not supported for pandas and the user should convert their data to xarray - to formulate the call to the metric. - - At most one of reduce_dims and preserve_dims may be specified. Specifying both will result in an exception. + fcst (Union[xr.Dataset, xr.DataArray, pd.Dataframe, pd.Series]): + Forecast or predicted variables in xarray or pandas. + obs (Union[xr.Dataset, xr.DataArray, pd.Dataframe, pd.Series]): + Observed variables in xarray or pandas. + reduce_dims (Union[str, Iterable[str]): Optionally specify which + dimensions to reduce when calculating MSE. All other dimensions + will be preserved. + preserve_dims (Union[str, Iterable[str]): Optionally specify which + dimensions to preserve when calculating MSE. All other dimensions + will be reduced. As a special case, 'all' will allow all dimensions + to be preserved. In this case, the result will be in the same + shape/dimensionality as the forecast, and the errors will be + the squared error at each point (i.e. single-value comparison + against observed), and the forecast and observed dimensions + must match precisely. + weights: Not yet implemented. Allow weighted averaging (e.g. by + area, by latitude, by population, custom) + + Returns: + Union[xr.Dataset, xr.DataArray, pd.Dataframe, pd.Series]: An object containing + a single floating point number representing the mean absolute + error for the supplied data. All dimensions will be reduced. + Otherwise: Returns an object representing the mean squared error, + reduced along the relevant dimensions and weighted appropriately. """ error = fcst - obs @@ -53,38 +60,40 @@ def mse(fcst, obs, reduce_dims=None, preserve_dims=None, weights=None): def mae(fcst, obs, reduce_dims=None, preserve_dims=None, weights=None): - """**Needs a 1 liner function description** + """Calculates the mean absolute error from forecast and observed data. + + A detailed explanation is on [Wikipedia](https://en.wikipedia.org/wiki/Mean_absolute_error) + + Dimensional reduction is not supported for pandas and the user should + convert their data to xarray to formulate the call to the metric. + At most one of reduce_dims and preserve_dims may be specified. + Specifying both will result in an exception. + Args: - - fcst: Forecast or predicted variables in xarray or pandas. - - obs: Observed variables in xarray or pandas. - - reduce_dims: Optionally specify which dimensions to reduce when - calculating MAE. All other dimensions will be preserved. - - preserve_dims: Optionally specify which dimensions to preserve - when calculating MAE. All other dimensions will be reduced. - As a special case, 'all' will allow all dimensions to be - preserved. In this case, the result will be in the same - shape/dimensionality as the forecast, and the errors will be - the absolute error at each point (i.e. single-value comparison - against observed), and the forecast and observed dimensions - must match precisely. - - weights: Not yet implemented. Allow weighted averaging (e.g. by - area, by latitude, by population, custom). + fcst (Union[xr.Dataset, xr.DataArray, pd.Dataframe, pd.Series]): Forecast + or predicted variables in xarray or pandas. + obs (Union[xr.Dataset, xr.DataArray, pd.Dataframe, pd.Series]): Observed + variables in xarray or pandas. + reduce_dims (Union[str, Iterable[str]]): Optionally specify which dimensions + to reduce when calculating MAE. All other dimensions will be preserved. + preserve_dims (Union[str, Iterable[str]]): Optionally specify which + dimensions to preserve when calculating MAE. All other dimensions + will be reduced. As a special case, 'all' will allow all dimensions + to be preserved. In this case, the result will be in the same + shape/dimensionality as the forecast, and the errors will be + the absolute error at each point (i.e. single-value comparison + against observed), and the forecast and observed dimensions + must match precisely. + weights: Not yet implemented. Allow weighted averaging (e.g. by + area, by latitude, by population, custom). Returns: - - By default an xarray DataArray containing a single floating - point number representing the mean absolute error for the + Union[xr.Dataset, xr.DataArray, pd.Dataframe, pd.Series]: By default an xarray DataArray containing + a single floating point number representing the mean absolute error for the supplied data. All dimensions will be reduced. - Alternatively, an xarray structure with dimensions preserved as - appropriate containing the score along reduced dimensions - - Notes: - - Dimensional reduction is not supported for pandas and the user - should convert their data to xarray to formulate the call to the metric. - - At most one of reduce_dims and preserve_dims may be specified. - Specifying both will result in an exception. - - A detailed explanation is on [Wikipedia](https://en.wikipedia.org/wiki/Mean_absolute_error) + Alternatively, an xarray structure with dimensions preserved as appropriate + containing the score along reduced dimensions """ error = fcst - obs diff --git a/src/scores/probability/__init__.py b/src/scores/probability/__init__.py index 1482bd01..92ab3b95 100644 --- a/src/scores/probability/__init__.py +++ b/src/scores/probability/__init__.py @@ -2,4 +2,8 @@ Import the functions from the implementations into the public API """ -from .crps_impl import adjust_fcst_for_crps, crps_cdf, crps_cdf_brier_decomposition +from scores.probability.crps_impl import ( + adjust_fcst_for_crps, + crps_cdf, + crps_cdf_brier_decomposition, +) diff --git a/src/scores/probability/checks.py b/src/scores/probability/checks.py index ad869d54..3b44a10e 100644 --- a/src/scores/probability/checks.py +++ b/src/scores/probability/checks.py @@ -1,5 +1,5 @@ """ -This module contains methods which make assertions at runtime about the state of various data +This module contains methods which make assertions at runtime about the state of various data structures and values """ @@ -8,24 +8,30 @@ def coords_increasing(da: xr.DataArray, dim: str): - """ - Returns True if coordinates along `dim` dimension of `da` are increasing, - False otherwise. No in-built raise if `dim` is not a dimension of `da`. + """Checks if coordinates in a given DataArray are increasing. + + Note: No in-built raise if `dim` is not a dimension of `da`. + + Args: + da (xr.DataArray): Input data + dim (str): Dimension to check if increasing + Returns: + (bool): Returns True if coordinates along `dim` dimension of + `da` are increasing, False otherwise. """ result = (da[dim].diff(dim) > 0).all() return result def cdf_values_within_bounds(cdf: xr.DataArray) -> bool: - """ - Checks that 0 <= cdf <= 1. Ignores NaNs. + """Checks that 0 <= cdf <= 1. Ignores NaNs. Args: - cdf: array of CDF values + cdf (xr.DataArray): array of CDF values Returns: - `True` if `cdf` values are all between 0 and 1 whenever values are not NaN, - or if all values are NaN; and `False` otherwise. + (bool): `True` if `cdf` values are all between 0 and 1 whenever values are not NaN, + or if all values are NaN; and `False` otherwise. """ return cdf.count() == 0 or ((cdf.min() >= 0) & (cdf.max() <= 1)) diff --git a/src/scores/probability/crps_impl.py b/src/scores/probability/crps_impl.py index 758e2949..b6309fc6 100644 --- a/src/scores/probability/crps_impl.py +++ b/src/scores/probability/crps_impl.py @@ -10,9 +10,8 @@ import xarray as xr import scores.utils - -from .checks import coords_increasing -from .functions import ( +from scores.probability.checks import coords_increasing +from scores.probability.functions import ( add_thresholds, cdf_envelope, decreasing_cdfs, @@ -157,7 +156,8 @@ def crps_cdf( weights=None, include_components=False, ): - """ + """Calculates CRPS CDF probabilistic metric. + Calculates the continuous ranked probability score (CRPS), or the mean CRPS over specified dimensions, given forecasts in the form of predictive cumulative distribution functions (CDFs). Can also calculate threshold-weighted versions of the @@ -174,31 +174,32 @@ def crps_cdf( - a predictive CDF `fcst` indexed at thresholds by variable x, - an observation in CDF form `obs_cdf` (i.e., obs_cdf(x) = 0 if x < obs and 1 if x >= obs), - a `threshold_weight` array indexed by variable x, - the threshold-weighted CRPS is given by - CRPS = integral(threshold_weight(x) * (fcst(x) - obs_cdf(x))**2), over all thresholds x. + + The threshold-weighted CRPS is given by: + `CRPS = integral(threshold_weight(x) * (fcst(x) - obs_cdf(x))**2)`, over all thresholds x. The usual CRPS is the threshold-weighted CRPS with `threshold_weight(x) = 1` for all x. This can be decomposed into an over-forecast penalty - integral(threshold_weight(x) * (fcst(x) - obs_cdf(x))**2), over all thresholds x where x >= obs + `integral(threshold_weight(x) * (fcst(x) - obs_cdf(x))**2)`, over all thresholds x where x >= obs and an under-forecast penalty - integral(threshold_weight(x) * (fcst(x) - obs_cdf(x))**2), over all thresholds x where x <= obs. + `integral(threshold_weight(x) * (fcst(x) - obs_cdf(x))**2)`, over all thresholds x where x <= obs. Note that the function `crps_cdf` is designed so that the `obs` argument contains actual observed values. `crps_cdf` will convert `obs` into CDF form in order to calculate the CRPS. To calculate CRPS, integration is applied over the set of thresholds x taken from - - `fcst[threshold_dim].values` - - `obs.values` - - `threshold_weight[threshold_dim].values` if applicable - - `additional_thresholds` if applicable - with NaN values excluded. There are two methods of integration: + - `fcst[threshold_dim].values`, + - `obs.values`. + - `threshold_weight[threshold_dim].values` if applicable. + - `additional_thresholds` if applicable. + With NaN values excluded. There are two methods of integration: - "exact" gives the exact integral under that assumption that that `fcst` is - continuous and piecewise linear between its specified values, and that - `threshold_weight` (if supplied) is piecewise constant and right-continuous - between its specified values. + continuous and piecewise linear between its specified values, and that + `threshold_weight` (if supplied) is piecewise constant and right-continuous + between its specified values. - "trapz" simply uses a trapezoidal rule using the specified values, and so is - an approximation of the CRPS. To get an accurate approximation, the density - of threshold values can be increased by supplying `additional_thresholds`. + an approximation of the CRPS. To get an accurate approximation, the density + of threshold values can be increased by supplying `additional_thresholds`. Both methods of calculating CRPS may require adding additional values to the `threshold_dim` dimension in `fcst` and (if supplied) `threshold_weight`. @@ -229,13 +230,13 @@ def crps_cdf( (by including additional thresholds) or are specified to be removed (by setting `propagate_nans=False`). Select one of: - "linear": use linear interpolation, then replace any leading or - trailing NaNs using linear extrapolation. Afterwards, all values are - clipped to the closed interval [0, 1]. + trailing NaNs using linear extrapolation. Afterwards, all values are + clipped to the closed interval [0, 1]. - "step": apply forward filling, then replace any leading NaNs with 0. - "forward": first apply forward filling, then remove any leading NaNs by - back filling. + back filling. - "backward": first apply back filling, then remove any trailing NaNs by - forward filling. + forward filling. In most cases, "linear" is likely the appropriate choice. threshold_weight_fill_method: how to fill values in `threshold_weight` when NaNs have been introduced (by including additional thresholds) or are specified @@ -243,19 +244,19 @@ def crps_cdf( "step", "forward" or "backward". If the weight function is continuous, "linear" is probably the best choice. If it is an increasing step function, "forward" may be best. - integration_method: one of "exact" or "trapz". - preserve_dims: dimensions to preserve in the output. All other dimensions are collapsed + integration_method (str): one of "exact" or "trapz". + preserve_dims (Tuple[str]): dimensions to preserve in the output. All other dimensions are collapsed by taking the mean. - reduce_dims: dimensions to reduce in the output by taking the mean. All other dimensions are + reduce_dims (Tuple[str]): dimensions to reduce in the output by taking the mean. All other dimensions are preserved. - - weights: Not yet implemented. Allow weighted averaging (e.g. by area, by latitude, by population, custom) - include_components: if True, include the under and over forecast components of + weights: Not yet implemented. Allow weighted averaging (e.g. by area, by latitude, by population, custom) + include_components (bool): if True, include the under and over forecast components of the score in the returned dataset. Returns: - an xarray Dataset with - - "total": the total CRPS - - "underforecast_penalty": the under-forecast penalty contribution of the CRPS + xr.Dataset: The following are the produced Dataset variables: + - "total" the total CRPS. + - "underforecast_penalty": the under-forecast penalty contribution of the CRPS. - "overforecast_penalty": the over-forecast penalty contribution of the CRPS. Raises: @@ -280,7 +281,6 @@ def crps_cdf( See also: `scores.probability.crps_cdf_brier_decomposition` - References: - Matheson, J. E., and R. L. Winkler, 1976: Scoring rules for continuous probability distributions. Manage. Sci.,22, 1087–1095. @@ -360,10 +360,10 @@ def crps_cdf_exact( """ Calculates exact value of CRPS assuming that: - the forecast CDF is continuous piecewise linear, with join points given by - values in `cdf_fcst`, + values in `cdf_fcst`, - the observation CDF is right continuous with values in {0,1} given by `cdf_obs`, - the threshold weight function is right continuous with values in {0,1} given - by `threshold_weight`. + by `threshold_weight`. If these assumptions do not hold, it might be best to use `crps_approximate`, with a sufficiently high resolution along `threshold_dim`. @@ -371,10 +371,11 @@ def crps_cdf_exact( This function assumes that `cdf_fcst`, `cdf_obs`, `threshold_weight` have same shape. Also assumes that values along the `threshold_dim` dimension are increasing. - Returns an xarray Dataset with `threshold_dim` collapsed containing DataArrays with - CRPS and its decomposition, labelled "total", "underforecast_penalty" and - "overforecast_penalty". NaN is returned if there is a NaN in the corresponding - `cdf_fcst`, `cdf_obs` or `threshold_weight`. + Returns: + (xr.Dataset): Dataset with `threshold_dim` collapsed containing DataArrays with + CRPS and its decomposition, labelled "total", "underforecast_penalty" and + "overforecast_penalty". NaN is returned if there is a NaN in the corresponding + `cdf_fcst`, `cdf_obs` or `threshold_weight`. """ # identify where input arrays have no NaN, collapsing `threshold_dim` @@ -442,22 +443,23 @@ def crps_cdf_brier_decomposition( `scores.probability.functions.fill_cdf`. Args: - fcst: DataArray of CDF values with threshold dimension `threshold_dim`. - obs: DataArray of observations, not in CDF form. - threshold_dim: name of the threshold dimension in `fcst`. - additional_thresholds: additional thresholds at which to calculate the mean - Brier score. - fcst_fill_method: How to fill NaN values in `fcst` that arise from new - user-supplied thresholds or thresholds derived from observations. + fcst (xr.DataArray): DataArray of CDF values with threshold dimension `threshold_dim`. + obs (xr.DataArray): DataArray of observations, not in CDF form. + threshold_dim (str): name of the threshold dimension in `fcst`. + additional_thresholds (Optional[Iterable[float]]): additional thresholds + at which to calculate the mean Brier score. + fcst_fill_method (Literal["linear", "step", "forward", "backward"]): How to fill NaN + values in `fcst` that arise from new user-supplied thresholds or thresholds derived + from observations. - "linear": use linear interpolation, and if needed also extrapolate linearly. - Clip to 0 and 1. Needs at least two non-NaN values for interpolation, - so returns NaNs where this condition fails. + Clip to 0 and 1. Needs at least two non-NaN values for interpolation, + so returns NaNs where this condition fails. - "step": use forward filling then set remaining leading NaNs to 0. - Produces a step function CDF (i.e. piecewise constant). + Produces a step function CDF (i.e. piecewise constant). - "forward": use forward filling then fill any remaining leading NaNs with - backward filling. + backward filling. - "backward": use backward filling then fill any remaining trailing NaNs with - forward filling. + forward filling. dims: dimensions to preserve in the output. The dimension `threshold_dim` is always preserved, even if not specified here. @@ -701,27 +703,27 @@ def crps_step_threshold_weight( steppoint_precision: float = 0, weight_upper: bool = True, ) -> xr.DataArray: - """ + """Generates an array of weights based on DataArray step points. + Creates an array of threshold weights, which can be used to calculate threshold-weighted CRPS, based on a step function. Applies a weight of 1 when step_point >= threshold, and a weight of 0 otherwise. Zeros and ones in the output weight function can be reversed by setting `weight_upper=False`. Args: - step_points: points at which the weight function changes value from 0 to 1. - threshold_dim: name of the threshold dimension in the returned array weight function. - threshold_values: thresholds at which to calculate weights. + step_points (xr.DataArray): points at which the weight function changes value from 0 to 1. + threshold_dim (str): name of the threshold dimension in the returned array weight function. + threshold_values (str): thresholds at which to calculate weights. steppoints_in_thresholds (bool): include `step_points` among the `threshold_dim` values. - steppoint_precision: precision at which to round step_points prior to calculating the + steppoint_precision (float): precision at which to round step_points prior to calculating the weight function. Select 0 for no rounding. - weight_upper: If true, returns a weight of 1 if step_point >= threshold, and a + weight_upper (bool): If true, returns a weight of 1 if step_point >= threshold, and a weight of 0 otherwise. If false, returns a weight of 0 if step_point >= threshold, and a weight of 1 otherwise. Returns: - an xarray DataArray of zeros and ones with the dimensions in `step_points` + (xr.DataArray): Zeros and ones with the dimensions in `step_points` and an additional `threshold_dim` dimension. - """ weight = observed_cdf( diff --git a/src/scores/probability/functions.py b/src/scores/probability/functions.py index e4e2187b..70f2abf8 100644 --- a/src/scores/probability/functions.py +++ b/src/scores/probability/functions.py @@ -9,24 +9,28 @@ import pandas as pd import xarray as xr -from .checks import cdf_values_within_bounds, check_nan_decreasing_inputs +from scores.probability.checks import ( + cdf_values_within_bounds, + check_nan_decreasing_inputs, +) def round_values(array: xr.DataArray, rounding_precision: float, final_round_decpl: int = 7) -> xr.DataArray: - """ - Round data array to specified precision. + """Round data array to specified precision. + Assumes that rounding_precision >=0, with 0 indicating no rounding to be performed. For example, 3.73 rounded to precision 0.2 is 3.8. If rounding_precision > 0, a final round to `final_round_decpl` decimal places is performed to remove artefacts of python rounding process. Args: - array: array of data to be rounded - rounding_precision: rounding precision - final_round_decpl: final round to specified number of decimal places when `rounding_precision` > 0. + array (xr.DataArray): array of data to be rounded + rounding_precision (float): rounding precision + final_round_decpl (int): final round to specified number of decimal + places when `rounding_precision` > 0. Returns: - An xarray data array with rounded values. + xr.DataArray: DataArray with rounded values. Raises: ValueError: If `rounding_precision` < 0. @@ -42,17 +46,15 @@ def round_values(array: xr.DataArray, rounding_precision: float, final_round_dec def propagate_nan(cdf: xr.DataArray, threshold_dim: str) -> xr.DataArray: - """ - If there is a NaN value in xr.DataArray `cdf`, then `propagate_nan` propagates the NaN value - along the `threshold_dim` dimension in both directions. + """Propagates the NaN values from a "cdf" variable along the `threshold_dim`. Args: - cdf: DataArray of CDF values, so that P(X <= threshold) = cdf_value for each threshold - in the `threshold_dim` dimension. - threshold_dim: name of the threshold dimension in `cdf`. + cdf (xr.DataArray): CDF values, so that P(X <= threshold) = cdf_value for + each threshold in the `threshold_dim` dimension. + threshold_dim (str): name of the threshold dimension in `cdf`. Returns: - An xarray DataArray of `cdf` but with NaNs propagated. + xr.DataArray: `cdf` variable with NaNs propagated. Raises: ValueError: If `threshold_dim` is not a dimension of `cdf`. @@ -72,27 +74,27 @@ def observed_cdf( include_obs_in_thresholds: bool = True, precision: float = 0, ) -> xr.DataArray: - """ - Returns a data array of observations converted into CDF format, such that + """Returns a data array of observations converted into CDF format. + + Such that: returned_value = 0 if threshold < observation returned_value = 1 if threshold >= observation Args: - obs: observations - threshold_dim: name of dimension in returned array that contains the threshold values. - threshold_values: values to include among thresholds. - include_obs_in_thresholds: if `True`, include (rounded) observed values among thresholds. - precision: precision applied to observed values prior to constructing the CDF and + obs (xr.DataArray): observations + threshold_dim (str): name of dimension in returned array that contains the threshold values. + threshold_values (Optional[Iterable[float]]): values to include among thresholds. + include_obs_in_thresholds (bool): if `True`, include (rounded) observed values among thresholds. + precision (float): precision applied to observed values prior to constructing the CDF and thresholds. Select 0 for highest precision (i.e. no rounding). Returns: - an xarray DataArray with observed CDFs and thresholds in the `threshold_dim` dimension. + xr.DataArray: Observed CDFs and thresholds in the `threshold_dim` dimension. Raises: ValueError: if `precision < 0`. ValueError: if all observations are NaN and no non-NaN `threshold_values` are not supplied. - """ if precision < 0: raise ValueError("`precision` must be nonnegative.") @@ -132,7 +134,8 @@ def observed_cdf( def integrate_square_piecewise_linear(function_values: xr.DataArray, threshold_dim: str) -> xr.DataArray: - """ + """Calculates integral values and collapses `threshold_dim`. + Calculates integral(F(t)^2), where - If t in a threshold value in `threshold_dim` then F(t) is in `function_values`, - F is piecewise linear between each of the t values in `threshold_dim`. @@ -144,11 +147,11 @@ def integrate_square_piecewise_linear(function_values: xr.DataArray, threshold_d - coordinates of `threshold_dim` are increasing. Args: - function_values: array of function values F(t). - threshold_dim: dimension along which to integrate. + function_values (xr.DataArray): array of function values F(t). + threshold_dim (xr.DataArray): dimension along which to integrate. Returns: - An xarray DataArray of integral values and `threshold_dim` collapsed. + xr.DataArray: Integral values and `threshold_dim` collapsed. """ # notation: Since F is piecewise linear we have @@ -183,22 +186,21 @@ def add_thresholds( fill_method: Literal["linear", "step", "forward", "backward", "none"], min_nonnan: int = 2, ) -> xr.DataArray: - """ - Takes a CDF data array with dimension `threshold_dim` and adds values from - `new_thresholds` to the `threshold_dim` dimension. + """Takes a CDF data array with dimension `threshold_dim` and adds values from `new_thresholds`. + The CDF is then filled to replace any NaN values. The array `cdf` requires at least 2 non-NaN values along `threshold_dim`. Args: - cdf: array of CDF values. - threshold_dim: name of the threshold dimension in `cdf`. - new_thresholds: new thresholds to add to `cdf`. - fill_method: one of "linear", "step", "forward" or "backward", as described in `fill_cdf`. - If no filling, set to "none". - min_nonnan: passed onto `fill_cdf` for performing filling. + cdf (xr.DataArray): array of CDF values. + threshold_dim (str): name of the threshold dimension in `cdf`. + new_thresholds (Iterable[float]): new thresholds to add to `cdf`. + fill_method (Literal["linear", "step", "forward", "backward", "none"]): one of "linear", + "step", "forward" or "backward", as described in `fill_cdf`. If no filling, set to "none". + min_nonnan (int): passed onto `fill_cdf` for performing filling. Returns: - an xarray DataArray with additional thresholds, and values at those thresholds + xr.DataArray: Additional thresholds, and values at those thresholds determined by the specified fill method. """ @@ -222,31 +224,28 @@ def fill_cdf( method: Literal["linear", "step", "forward", "backward"], min_nonnan: int, ) -> xr.DataArray: - """ - Fills NaNs in a CDF of a real-valued random variable along `threshold_dim` - with appropriate values between 0 and 1. + """Fills NaNs in a CDF of a real-valued random variable along `threshold_dim` with appropriate values between 0 and 1. Args: - cdf: CDF values, where P(Y <= threshold) = cdf_value for each threshold in `threshold_dim`. - threshold_dim: the threshold dimension in the CDF, along which filling is performed. - method: one of + cdf (xr.DataArray): CDF values, where P(Y <= threshold) = cdf_value for each threshold in `threshold_dim`. + threshold_dim (str): the threshold dimension in the CDF, along which filling is performed. + method (Literal["linear", "step", "forward", "backward"]): one of - "linear": use linear interpolation, and if needed also extrapolate linearly. Clip to 0 and 1. - Needs at least two non-NaN values for interpolation, so returns NaNs where this condition fails. + Needs at least two non-NaN values for interpolation, so returns NaNs where this condition fails. - "step": use forward filling then set remaining leading NaNs to 0. - Produces a step function CDF (i.e. piecewise constant). + Produces a step function CDF (i.e. piecewise constant). - "forward": use forward filling then fill any remaining leading NaNs with backward filling. - "backward": use backward filling then fill any remaining trailing NaNs with forward filling. - min_nonnan: the minimum number of non-NaN entries required along `threshold_dim` for filling to + min_nonnan (int): the minimum number of non-NaN entries required along `threshold_dim` for filling to be performed. All CDF values are set to `np.nan` where this condition fails. `min_nonnan` must be at least 2 for the "linear" method, and at least 1 for the other methods. Returns: - An xarray DataArray with the same values as `cdf` but with NaNs filled. + xr.DataArray: Containing the same values as `cdf` but with NaNs filled. Raises: ValueError: If `threshold_dim` is not a dimension of `cdf`. - ValueError: If `min_nonnan` < 1 when `method="step"` - or if `min_nonnan` < 2 when `method="linear"`. + ValueError: If `min_nonnan` < 1 when `method="step"` or if `min_nonnan` < 2 when `method="linear"`. ValueError: If `method` is not "linear", "step", "forward" or "backward". ValueError: If any non-NaN value of `cdf` lies outside the unit interval [0,1]. @@ -290,8 +289,8 @@ def fill_cdf( def decreasing_cdfs(cdf: xr.DataArray, threshold_dim: str, tolerance: float) -> xr.DataArray: - """ - A CDF of a real-valued random variable should be nondecreasing along threshold_dim. + """A CDF of a real-valued random variable should be nondecreasing along threshold_dim. + This is sometimes violated due to rounding issues or bad forecast process. `decreasing_cdfs` checks CDF values decrease beyond specified tolerance; that is, whenever the sum of the incremental decreases exceeds tolerarance. @@ -304,16 +303,15 @@ def decreasing_cdfs(cdf: xr.DataArray, threshold_dim: str, tolerance: float) -> either each CDF is always NaN or always non-NaN. Args: - cdf: data array of CDF values - threshold_dim: threshold dimension, such that P(Y < threshold) = cdf_value. - tolerance: nonnegative tolerance value. + cdf (xr.DataArray): data array of CDF values + threshold_dim (str): threshold dimension, such that P(Y < threshold) = cdf_value. + tolerance (float): nonnegative tolerance value. Returns: - An xarray DataArray with `threshold_dim` collapsed and values True if and only if + xr.DataArray: Containing `threshold_dim` collapsed and values True if and only if the CDF is decreasing outside tolerance. If the CDF consists only of NaNs then the value is False. - Raises: ValueError: If `threshold_dim` is not a dimension of `cdf`. ValueError: If `tolerance` is negative. @@ -334,13 +332,12 @@ def cdf_envelope( cdf: xr.DataArray, threshold_dim: str, ) -> xr.DataArray: - """ - Forecast cumulative distribution functions (CDFs) for real-valued random variables - that are reconstructed from known points on the - distribution should be nondecreasing with respect to the threshold dimension. - However, sometimes this may fail due to rounding or poor forecast process. - This function returns the "envelope" of the original CDF, which consists of - two bounding CDFs, both of which are nondecreasing. + """Forecast cumulative distribution functions (CDFs) for real-valued random variables. + + CDFs that are reconstructed from known points on the distribution should be nondecreasing + with respect to the threshold dimension. However, sometimes this may fail due to rounding + or poor forecast process. This function returns the "envelope" of the original CDF, which + consists of two bounding CDFs, both of which are nondecreasing. The following example shows values from an original CDF that has a decreasing subsequence (and so is not a true CDF). The resulting "upper" and "lower" CDFs minimally adjust @@ -352,17 +349,17 @@ def cdf_envelope( This function does not perform checks that `0 <= cdf <= 1`. Args: - cdf: forecast CDF with thresholds in the thresholds_dim. - threshold_dim: dimension in fcst_cdf that contains the threshold ordinates. + cdf (xr.DataArray): forecast CDF with thresholds in the thresholds_dim. + threshold_dim (str): dimension in fcst_cdf that contains the threshold ordinates. Returns: An xarray DataArray consisting of three CDF arrays indexed along the `"cdf_type"` dimension with the following indices: - "original": same data as `cdf`. - "upper": minimally adjusted "original" CDF that is nondecreasing and - satisfies "upper" >= "original". + satisfies "upper" >= "original". - "lower": minimally adjusted "original" CDF that is nondecreasing and - satisfies "lower" <= "original". + satisfies "lower" <= "original". NaN values in `cdf` are maintained in "original", "upper" and "lower". Raises: diff --git a/src/scores/sample_data.py b/src/scores/sample_data.py index 72a7ae95..6b3e40bf 100644 --- a/src/scores/sample_data.py +++ b/src/scores/sample_data.py @@ -19,13 +19,14 @@ def simple_observations() -> xr.DataArray: def continuous_observations(large_size: bool = False) -> xr.DataArray: - """ - Creates a obs array with continuous values. + """Creates a obs array with continuous values. + Args: - large_size: If True, then returns a large global array with ~0.5 degree + large_size (bool): If True, then returns a large global array with ~0.5 degree grid spacing, otherwise returns a cut down, lower resolution array. - Returns an xr.Datarray with synthetic observation data. + Returns: + xr.Datarray: Containing synthetic observation data. """ num_lats = 10 @@ -53,15 +54,15 @@ def continuous_observations(large_size: bool = False) -> xr.DataArray: def continuous_forecast(large_size: bool = False, lead_days: bool = False) -> xr.DataArray: - """ - Creates a forecast array with continuous values. + """Creates a forecast array with continuous values. Args: - large_size: If True, then returns a large global array with ~0.5 degree + large_size (bool): If True, then returns a large global array with ~0.5 degree grid spacing, otherwise returns a cut down, lower resolution array. - lead_days: If True, returns an array with a "lead_day" dimension. + lead_days (bool): If True, returns an array with a "lead_day" dimension. - Returns an xr.Datarray with synthetic forecast data. + Returns: + xr.Datarray: Containing synthetic forecast data. """ obs = continuous_observations(large_size) np.random.seed(42) @@ -78,9 +79,10 @@ def cdf_forecast(lead_days: bool = False) -> xr.DataArray: Creates a forecast array with a CDF at each point. Args: - lead_days: If True, returns an array with a "lead_day" dimension. + lead_days (bool): If True, returns an array with a "lead_day" dimension. - Returns an xr.Datarray with synthetic CDF forecast data. + Returns: + xr.Datarray: Containing synthetic CDF forecast data. """ x = np.arange(0, 10, 0.1) cdf_list = [] @@ -120,7 +122,8 @@ def cdf_observations() -> xr.DataArray: """ Creates an obs array to use with `cdf_forecast`. - Returns an xr.Datarray with synthetic observations betwen 0 and 9.9 + Returns: + xr.Datarray: Containing synthetic observations betwen 0 and 9.9 """ np.random.seed(42) obs = xr.DataArray( diff --git a/src/scores/utils.py b/src/scores/utils.py index ecc72eed..4ce4828e 100644 --- a/src/scores/utils.py +++ b/src/scores/utils.py @@ -35,9 +35,22 @@ class DimensionError(Exception): Dataset objects that do not have compatible dimensions. """ + def gather_dimensions(fcst_dims, obs_dims, weights_dims=None, reduce_dims=None, preserve_dims=None): """ Establish which dimensions to reduce when calculating errors but before taking means + + Args: + fcst_dims (Iterable[str]): Forecast dimensions inputs + obs_dims (Iterable[str]): Observation dimensions inputs. + weights_dims (Iterable[str]): Weight dimension inputs. + reduce_dims (Union[str, Iterable[str]]): Dimensions to reduce. + preserve_dims (Union[str, Iterable[str]]): Dimensions to preserve. + + Returns: + Tuple[str]: Dimensions based on optional args. + Raises: + ValueError: When `preserve_dims and `reduce_dims` are both specified. """ all_dims = set(fcst_dims).union(set(obs_dims)) @@ -89,15 +102,14 @@ def gather_dimensions(fcst_dims, obs_dims, weights_dims=None, reduce_dims=None, def dims_complement(data, dims=None): - """ - Returns the complement of data.dims and dims + """Returns the complement of data.dims and dims Args: - data: an xarray DataArray or Dataset - dims: an Iterable of strings corresponding to dimension names + data (Union[xr.Dataset, xr.DataArray]): Input xarray object + dims (Iterable[str]): an Iterable of strings corresponding to dimension names Returns: - A sorted list of dimension names, the complement of data.dims and dims + List[str]: A sorted list of dimension names, the complement of data.dims and dims """ if dims is None: dims = [] @@ -114,37 +126,33 @@ def check_dims(xr_data, expected_dims, mode=None): Checks the dimensions xr_data with expected_dims, according to `mode`. Args: - xr_data (xarray.DataArray or xarray.Dataset): if a Dataset is supplied, + xr_data (Union[xarray.DataArray, xr.Dataset]): if a Dataset is supplied, all of its data variables (DataArray objects) are checked. - expected_dims (Iterable): an Iterable of dimension names. + expected_dims (Iterable[str]): an Iterable of dimension names. mode (Optional[str]): one of 'equal' (default), 'subset' or 'superset'. - - - If 'equal', checks that the data object has the same dimensions - as `expected_dims`. - - If 'subset', checks that the dimensions of the data object is a - subset of `expected_dims`. - - If 'superset', checks that the dimensions of the data object is a - superset of `expected_dims`, (i.e. contains `expected_dims`). - - If 'proper subset', checks that the dimensions of the data object is a - subset of `expected_dims`, (i.e. is a subset, but not equal to - `expected_dims`). - - If 'proper superset', checks that the dimensions of the data object - is a proper superset of `expected_dims`, (i.e. contains but is not - equal to `expected_dims`). - - If 'disjoint', checks that the dimensions of the data object shares no - elements with `expected_dims`. - - Returns: - None + If 'equal', checks that the data object has the same dimensions + as `expected_dims`. + If 'subset', checks that the dimensions of the data object is a + subset of `expected_dims`. + If 'superset', checks that the dimensions of the data object is a + superset of `expected_dims`, (i.e. contains `expected_dims`). + If 'proper subset', checks that the dimensions of the data object is a + subset of `expected_dims`, (i.e. is a subset, but not equal to + `expected_dims`). + If 'proper superset', checks that the dimensions of the data object + is a proper superset of `expected_dims`, (i.e. contains but is not + equal to `expected_dims`). + If 'disjoint', checks that the dimensions of the data object shares no + elements with `expected_dims`. Raises: scores.utils.DimensionError: the dimensions of `xr_data` does not pass the check as specified by `mode`. - TypeError: `xr_data` is not an xarray data object - ValueError: `expected_dims` contains duplicate values - ValueError: `expected_dims` cannot be coerced into a set - ValueError: `mode` is not one of 'equal', 'subset', - 'superset', 'proper subset', 'proper superset', or 'disjoint' + TypeError: `xr_data` is not an xarray data object. + ValueError: `expected_dims` contains duplicate values. + ValueError: `expected_dims` cannot be coerced into a set. + ValueError: `mode` is not one of 'equal', 'subset', 'superset', + 'proper subset', 'proper superset', or 'disjoint' """