Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MNT, DOC] Accelerating deep testing #1904

Open
wants to merge 11 commits into
base: main
Choose a base branch
from

Conversation

hadifawaz1999
Copy link
Member

@hadifawaz1999 hadifawaz1999 commented Aug 4, 2024

Fix #1761

  • Changes test params of deep classifiers/regressors/clusterers so that testing is faster
  • In deep specific tests, use the test parameters, as they were not used before, to accelerate testing
  • Fix random state deep clustering test, it was looping infinitely before because there was a bug, it was considering the encoder/decoder as layers before, not its fixed and ensures encoders and decoders are testing separately
  • Add possibility to use dummy clusterer in deep clustering, for the clustering in latent space, use it as test parameter to accelerate testing especially in no numba label. Another reason for this addition is that with test parameters of AE based clusterers in deep learning, if not trained long enough, kmeans may not converge and CI will fail with "ValueError: Unable to find a valid cluster configuration with parameters specified (empty clusters kept forming). Try lowering your n_clusters or raising n_init." so it will be a pain to find the best seed, so replacing with dummy clusterer is best option
  • Fix Encoder Network issue with Lambda layer, using now internal attention layer, as the original authors implemented the network before keras introduced Attention layers. The math isnt 100% the same but the idea of attention is the same
  • Doc issue with AEResNet

@aeon-actions-bot aeon-actions-bot bot added clustering Clustering package documentation Improvements or additions to documentation maintenance Continuous integration, unit testing & package distribution networks Networks package labels Aug 4, 2024
@aeon-actions-bot
Copy link
Contributor

Thank you for contributing to aeon

I have added the following labels to this PR based on the title: [ $\color{#F3B9F8}{\textsf{documentation}}$, $\color{#EC843A}{\textsf{maintenance}}$ ].
I have added the following labels to this PR based on the changes made: [ $\color{#4011F3}{\textsf{clustering}}$, $\color{#379E11}{\textsf{networks}}$ ]. Feel free to change these if they do not properly represent the PR.

The Checks tab will show the status of our automated tests. You can click on individual test runs in the tab or "Details" in the panel below to see more information if there is a failure.

If our pre-commit code quality check fails, any trivial fixes will automatically be pushed to your PR unless it is a draft.

Don't hesitate to ask questions on the aeon Slack channel if you have any.

PR CI actions

These checkboxes will add labels to enable/disable CI functionality for this PR. This may not take effect immediately, and a new commit may be required to run the new configuration.

  • Run pre-commit checks for all files
  • Run all pytest tests and configurations
  • Run all notebook example tests
  • Run numba-disabled codecov tests
  • Stop automatic pre-commit fixes (always disabled for drafts)
  • Push an empty commit to re-run CI checks

@hadifawaz1999 hadifawaz1999 added the testing Testing related issue or pull request label Aug 5, 2024
Comment on lines +119 to +123
if self.clustering_algorithm == "dummy":
self.clusterer = DummyClusterer(
n_clusters=self.n_clusters, **clustering_params_
)
elif self.clustering_algorithm == "kmeans":
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this not just accept any BaseClusterer? Creating a useless option solely for testing is not a great way to resolve this IMO.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i wanted to change that for accepting an estimator input instead of string, but thought it might be a lot for the PR, but to keep the PR for testing purpose this can be done, if you think its ok to get all in one PR i dont mind can do the changes here

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't mind if you do it here. The dummy option is not a good addition IMO.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will add the changes then

@@ -138,36 +138,43 @@ def build_network(self, input_shape, **kwargs):
self._kernel_size_ = [8, 5, 3] if self.kernel_size is None else self.kernel_size

if isinstance(self._n_filters_, list):
assert len(self._n_filters_) == self.n_residual_blocks
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

raise an actual error with a message instead of asserting

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will do, we should raise an issue to do that all over the networks module, my code my bad ! never thought about raising a message

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be better as a ValueError IMO

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
clustering Clustering package documentation Improvements or additions to documentation maintenance Continuous integration, unit testing & package distribution networks Networks package testing Testing related issue or pull request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[MNT] testing random state of deep clustering module is slow
2 participants