Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wrong EXCEPTION_MESSAGE_WRONG_RETURN_TYPE_FIT message #3948

Open
LorenzoColombi opened this issue Jul 31, 2024 · 0 comments
Open

Wrong EXCEPTION_MESSAGE_WRONG_RETURN_TYPE_FIT message #3948

LorenzoColombi opened this issue Jul 31, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@LorenzoColombi
Copy link

Describe the bug

In client/numpy_client.py the EXCEPTION_MESSAGE_WRONG_RETURN_TYPE_FIT constant is set to:

  EXCEPTION_MESSAGE_WRONG_RETURN_TYPE_FIT = """
  NumPyClient.fit did not return a tuple with 3 elements.
  The returned values should have the following type signature:
  
      Tuple[NDArrays, int, Dict[str, Scalar]]
  
  Example
  -------
  
      model.get_weights(), 10, {"accuracy": 0.95}
  
  """

However, function _fit(...), that checks returned value type wants a Tuple[List, int, Dict[str,Scalar]]

def _fit(self: Client, ins: FitIns) -> FitRes:
    """Refine the provided parameters using the locally held dataset."""
    # Deconstruct FitIns
    parameters: NDArrays = parameters_to_ndarrays(ins.parameters)

    # Train
    results = self.numpy_client.fit(parameters, ins.config)  # type: ignore
    if not (
        len(results) == 3
        and isinstance(results[0], list)
        and isinstance(results[1], int)
        and isinstance(results[2], dict)
    ):
        print(results)
        print(len(results))
        
        raise TypeError(EXCEPTION_MESSAGE_WRONG_RETURN_TYPE_FIT)

Therefore, the error message printed is misleading for the users.

Steps/Code to Reproduce

To reproduce you simply need to implements a fit(...) function inside the numpy_client that returns a tuple with signature Tuple[NDArrays, int, Dict[str, Scalar]] as indicated in the error string.

For example:

    class MnistClient(fl.client.NumPyClient):
        def __init__(self, model, X_train) -> None:
            super().__init__()
            self.model = model
            self.X_train = X_train

        def get_parameters(self, config):  # type: ignore
            return utils.get_model_parameters(self.model)

        def fit(self, parameters, config):  # type: ignore
            utils.set_model_params(model, parameters)
            # Ignore convergence failure due to low local epochs
            with warnings.catch_warnings():
                warnings.simplefilter("ignore")
                self.model.fit(self.X_train, y_train)
            print(f"Training finished for round {config['server_round']}")

            params = model.coef_
            # instead of: params = [model.coef_]

            return params, len(X_train), {}

        def evaluate(self, parameters, config):  # type: ignore
            utils.set_model_params(model, parameters)
            loss = log_loss(y_test, model.predict_proba(X_test))
            accuracy = model.score(X_test, y_test)
            return loss, len(X_test), {"accuracy": accuracy}

Expected Results

Returning Tuple[NDArrays, int, Dict[str, Scalar]] should work as written in the error message, or the string should be fixed.

Actual Results

Returning a Tuple[NDArrays, int, Dict[str, Scalar]] in the fit function results in a raised exception and then an error.

@LorenzoColombi LorenzoColombi added the bug Something isn't working label Jul 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant