Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mypy checks for NNCF common pruning code #2613

Open
wants to merge 14 commits into
base: develop
Choose a base branch
from

Conversation

RitikaxShakya
Copy link

@RitikaxShakya RitikaxShakya commented Mar 30, 2024

Changes

This PR closes issue #2494 by addressing various mypy checks

  • Resolved specific mypy errors related to type inconsistencies.
  • Utilized # type:ignore for cases requiring significant refactoring due to untyped packages, attributes not defined etc.
  • Added the directory nncf/common/pruning to .mypy.ini to include additional files for type checking.

image
image
image

Tests

Pytest and make pre-commit test were run to ensure that the changes did not change any logic and that the codebase remained functional.

@RitikaxShakya RitikaxShakya requested a review from a team as a code owner March 30, 2024 11:48
@github-actions github-actions bot added the NNCF Common Pull request that updates NNCF Common label Mar 30, 2024
@RitikaxShakya
Copy link
Author

@vshampor @p-wysocki Hello! Please take a look at my PR and let me know if i would need to do anything more. Thank you :)

@openvino-nncf-ci openvino-nncf-ci added the API Public API-impacting changes label Mar 30, 2024
Copy link

codecov bot commented Mar 30, 2024

Codecov Report

Attention: Patch coverage is 34.30657% with 90 lines in your changes are missing coverage. Please review.

Project coverage is 28.01%. Comparing base (fa8b702) to head (159e2dd).
Report is 10 commits behind head on develop.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff              @@
##           develop    #2613       +/-   ##
============================================
- Coverage    91.17%   28.01%   -63.17%     
============================================
  Files          494      494               
  Lines        45428    45379       -49     
============================================
- Hits         41420    12713    -28707     
- Misses        4008    32666    +28658     
Files Coverage Δ
nncf/common/pruning/clusterization.py 30.88% <88.88%> (-66.18%) ⬇️
nncf/common/pruning/model_analysis.py 24.46% <77.77%> (-74.47%) ⬇️
nncf/common/pruning/symbolic_mask.py 44.44% <75.00%> (-55.56%) ⬇️
nncf/common/pruning/node_selector.py 20.00% <0.00%> (-78.74%) ⬇️
nncf/common/pruning/utils.py 35.35% <56.25%> (-59.60%) ⬇️
nncf/common/pruning/shape_pruning_processor.py 29.41% <18.18%> (-68.21%) ⬇️
nncf/common/pruning/weights_flops_calculator.py 23.07% <0.00%> (-75.00%) ⬇️
nncf/common/pruning/mask_propagation.py 30.30% <21.42%> (-69.70%) ⬇️
nncf/common/pruning/operations.py 46.91% <31.57%> (-50.22%) ⬇️
nncf/common/pruning/schedulers.py 18.07% <9.09%> (-80.70%) ⬇️

... and 386 files with indirect coverage changes

Flag Coverage Δ
COMMON ?
ONNX ?
OPENVINO ?
TENSORFLOW 14.20% <34.30%> (-15.90%) ⬇️
TORCH 24.01% <34.30%> (-41.91%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
common 33.66% <34.30%> (-60.14%) ⬇️
torch 30.18% <ø> (-63.31%) ⬇️
tensorflow 30.62% <ø> (-63.13%) ⬇️
onnx 0.00% <ø> (-93.10%) ⬇️
openvino 0.00% <ø> (-94.17%) ⬇️
ptq 18.67% <ø> (-71.45%) ⬇️

@daniil-lyakhov
Copy link
Collaborator

Greeting, @RitikaxShakya!
Looks like some pre-commit checks haven't been passed, please try make test-torch on your local machine

@RitikaxShakya
Copy link
Author

@daniil-lyakhov Hello!
I passed make test-torch on my local machine and pushed again
The errors were same on both branch develop and issue#2494

image

@daniil-lyakhov
Copy link
Collaborator

@daniil-lyakhov Hello! I passed make test-torch on my local machine and pushed again The errors were same on both branch develop and issue#2494

image

Looks like an environment problem, our internal CI fails with following error:

tests/torch/accuracy_aware_training/test_accuracy_aware_config.py:16: in <module>
    from nncf.torch.initialization import register_default_init_args
nncf/torch/__init__.py:42: in <module>
    from nncf.torch.pruning.filter_pruning import algo as filter_pruning_algo
nncf/torch/pruning/filter_pruning/algo.py:31: in <module>
    from nncf.common.pruning.schedulers import PRUNING_SCHEDULERS
nncf/common/pruning/schedulers.py:30: in <module>
    class PruningScheduler(BaseCompressionScheduler):
nncf/common/pruning/schedulers.py:45: in PruningScheduler
    def __init__(self, controller: CompressionAlgorithmController, params: dict[str, Any]):
E   TypeError: 'type' object is not subscriptable

please try to reinstall nncf and launch torch tests

@RitikaxShakya
Copy link
Author

RitikaxShakya commented Apr 6, 2024

@daniil-lyakhov Hello! I reinstalled and run all tests Pytest , make pre-commit and make test-torch on my local machine and pushed again. For make pre-commit , all tests were passed, but for make test-torch, the errors are same as before.

@daniil-lyakhov
Copy link
Collaborator

@daniil-lyakhov Hello! I reinstalled and run all tests Pytest , make pre-commit and make test-torch on my local machine and pushed again. For make pre-commit , all tests were passed, but for make test-torch, the errors are same as before.

Hello! Sorry for a delay, but as the error states your python interpreter has no site packages directory (please see following call https://github.com/openvinotoolkit/nncf/blob/develop/tests/shared/paths.py#L23). Could you please try to run following commands to check your python is working correctly?

python3.8 -m venv nncf_env
source nncf_env/bin/activate
cd nncf/
make install-torch-test

@RitikaxShakya
Copy link
Author

RitikaxShakya commented Apr 10, 2024

@daniil-lyakhov Hello! Thank you! There were some packages missing, after running those commands and after make test-torch , I get these errors, but make pre-commit test were all passed :

ERROR tests/torch/accuracy_aware_training/test_accuracy_aware_config.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/accuracy_aware_training/test_runner.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/accuracy_aware_training/test_training_loop.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/automl/test_quantization_env.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/composite/test_sparsity_quantization.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/experimental/search_building_blocks/test_search_building_blocks.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/experimental/search_building_blocks/test_transformer_blocks.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/models_hub_test/test_hf_transformers.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/models_hub_test/test_timm.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/models_hub_test/test_torchvision_models.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/modules/test_rnn.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/nas/test_all_elasticity.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/nas/test_elastic_depth.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/nas/test_elastic_kernel.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/nas/test_elastic_width.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/nas/test_flops.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/nas/test_onnx_export.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/nas/test_ps_controller.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/nas/test_sanity_sample.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/nas/test_scheduler.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/nas/test_search.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/nas/test_search_space.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/nas/test_state.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/pruning/experimental/test_nodes_grouping.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/pruning/filter_pruning/test_algo.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/pruning/filter_pruning/test_evolutionary_optimization.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/pruning/filter_pruning/test_flops_pruning.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/pruning/filter_pruning/test_layers.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/pruning/filter_pruning/test_legr.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/pruning/filter_pruning/test_set_pruning_rate.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/pruning/filter_pruning/test_strip.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/pruning/test_common.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/pruning/test_distributed.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/pruning/test_model_pruning_analysis.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/pruning/test_onnx_export.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/pruning/test_schedulers.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/pruning/test_utils.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/ptq/test_calculation_quantizer_params.py
ERROR tests/torch/ptq/test_fast_bias_correction.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/ptq/test_fq_params_calculation.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/ptq/test_graphs.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/ptq/test_min_max.py
ERROR tests/torch/ptq/test_ptq_params.py
ERROR tests/torch/ptq/test_quantizer_config.py
ERROR tests/torch/ptq/test_smooth_quant.py
ERROR tests/torch/ptq/test_strip.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/ptq/test_weights_compression.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/ptq/test_wrap_model.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/qat/test_qat_classification.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/qat/test_qat_object_detection.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/qat/test_qat_segmentation.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/quantization/test_adjust_padding.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/quantization/test_algo_quantization.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/quantization/test_autoq_precision_init.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/quantization/test_functions.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/quantization/test_hawq_precision_init.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/quantization/test_hw_config.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/quantization/test_logarithm_scale.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/quantization/test_manual_precision_init.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/quantization/test_onnx_export.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/quantization/test_overflow_issue_export.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/quantization/test_quantization_metric.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/quantization/test_range_init.py
ERROR tests/torch/quantization/test_scheduler.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/quantization/test_strip.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/quantization/test_unified_scales.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/sparsity/const/test_algo.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/sparsity/magnitude/test_algo.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/sparsity/magnitude/test_helpers.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/sparsity/magnitude/test_modules.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/sparsity/magnitude/test_scheduler.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/sparsity/movement/test_algo.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/sparsity/movement/test_components.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/sparsity/movement/test_model_saving.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/sparsity/movement/test_scheduler.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/sparsity/movement/test_structured_mask.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/sparsity/movement/test_training.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/sparsity/rb/test_algo.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/sparsity/test_common.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/sparsity/test_strip.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_algo_common.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_api_behavior.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_backward_compat.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_compressed_graph.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_compression_lr_multiplier.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_compression_training.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_context_independence.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_custom_modules.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_distributed_data_parallel_mode.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_extractor.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_frozen_layers.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_graph_building.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_init_data_loader.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_input_management.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_knowledge_distillation.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_layer_attributes.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_load_model_state.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_model_analyzer.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_model_graph_manager.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_model_transformer.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_nncf_network.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_no_compression_algorithm.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_no_nncf_trace_patching.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_onnx_export.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_pytorch_patch.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_resume_from_checkpoint.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_sanity_sample.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_sota_checkpoints.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_statistics_aggregator.py
ERROR tests/torch/test_telemetry.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_tracing_context.py - TypeError: 'type' object is not subscriptable
ERROR tests/torch/test_utils.py - TypeError: 'type' object is not subscriptable
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 112 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
========================================== 28 deselected, 3 warnings, 112 errors in 7.91s ===========================================
make: *** [Makefile:146: test-torch] Error 2```

@daniil-lyakhov
Copy link
Collaborator

daniil-lyakhov commented Apr 12, 2024

@RitikaxShakya , Let's take a look on one of the error:

tests/torch/accuracy_aware_training/test_runner.py:23: in <module>
    from nncf.torch.accuracy_aware_training.runner import PTAccuracyAwareTrainingRunner
nncf/torch/__init__.py:42: in <module>
    from nncf.torch.pruning.filter_pruning import algo as filter_pruning_algo
nncf/torch/pruning/filter_pruning/algo.py:31: in <module>
    from nncf.common.pruning.schedulers import PRUNING_SCHEDULERS
nncf/common/pruning/schedulers.py:30: in <module>
    class PruningScheduler(BaseCompressionScheduler):
nncf/common/pruning/schedulers.py:45: in PruningScheduler
    def __init__(self, controller: CompressionAlgorithmController, params: dict[str, Any]):
E   TypeError: 'type' object is not subscriptable

here you are trying to use dict as a type, but it brakes python as dict is actual a type and it is not subscriptable. Please do this here and elsewhere

from typing import Dict
...
def __init__(self, controller: CompressionAlgorithmController, params: Dict[str, Any]):
...

@alexsu52 alexsu52 requested a review from daniil-lyakhov April 18, 2024 07:32
@RitikaxShakya
Copy link
Author

@RitikaxShakya , Let's take a look on one of the error:

tests/torch/accuracy_aware_training/test_runner.py:23: in <module>
    from nncf.torch.accuracy_aware_training.runner import PTAccuracyAwareTrainingRunner
nncf/torch/__init__.py:42: in <module>
    from nncf.torch.pruning.filter_pruning import algo as filter_pruning_algo
nncf/torch/pruning/filter_pruning/algo.py:31: in <module>
    from nncf.common.pruning.schedulers import PRUNING_SCHEDULERS
nncf/common/pruning/schedulers.py:30: in <module>
    class PruningScheduler(BaseCompressionScheduler):
nncf/common/pruning/schedulers.py:45: in PruningScheduler
    def __init__(self, controller: CompressionAlgorithmController, params: dict[str, Any]):
E   TypeError: 'type' object is not subscriptable

here you are trying to use dict as a type, but it brakes python as dict is actual a type and it is not subscriptable. Please do this here and elsewhere

from typing import Dict
...
def __init__(self, controller: CompressionAlgorithmController, params: Dict[str, Any]):
...

Hello @daniil-lyakhov Sorry for so late reply. I made the changes as you suggested seems like its still not passing checks. Also the page for details of NNCF TF precommit (Ubuntu 20.04), NNCF Torch + common precommit are not opening.

@daniil-lyakhov
Copy link
Collaborator

@RitikaxShakya , Let's take a look on one of the error:

tests/torch/accuracy_aware_training/test_runner.py:23: in <module>
    from nncf.torch.accuracy_aware_training.runner import PTAccuracyAwareTrainingRunner
nncf/torch/__init__.py:42: in <module>
    from nncf.torch.pruning.filter_pruning import algo as filter_pruning_algo
nncf/torch/pruning/filter_pruning/algo.py:31: in <module>
    from nncf.common.pruning.schedulers import PRUNING_SCHEDULERS
nncf/common/pruning/schedulers.py:30: in <module>
    class PruningScheduler(BaseCompressionScheduler):
nncf/common/pruning/schedulers.py:45: in PruningScheduler
    def __init__(self, controller: CompressionAlgorithmController, params: dict[str, Any]):
E   TypeError: 'type' object is not subscriptable

here you are trying to use dict as a type, but it brakes python as dict is actual a type and it is not subscriptable. Please do this here and elsewhere

from typing import Dict
...
def __init__(self, controller: CompressionAlgorithmController, params: Dict[str, Any]):
...

Hello @daniil-lyakhov Sorry for so late reply. I made the changes as you suggested seems like its still not passing checks. Also the page for details of NNCF TF precommit (Ubuntu 20.04), NNCF Torch + common precommit are not opening.

Hi @RitikaxShakya, no problems! Please try to rebase your PR. Besides following error appeared in TF pre-commit

   def generate_output_mask(
        cls, node: NNCFNode, graph: NNCFGraph, tensor_processor: Type[NNCFPruningBaseTensorProcessor]
    ) -> Optional[NNCFTensor]:
        """
        Generate output mask from input masks with all None replaced by identity masks.
        If all input masks is None return None.
    
        :param node: Node to determine it's sources.
        :param graph: NNCF graph to work with.
        :param tensor_processor: Interface with tensor processing methods.
        :return: Filled input masks.
        """
        input_edges = graph.get_input_edges(node)
        previous_nodes = [edge.from_node for edge in input_edges]
        input_masks = [input_node.attributes["output_mask"] for input_node in previous_nodes]
        input_masks = [
            input_mask[node.node_name] if isinstance(input_mask, dict) else input_mask for input_mask in input_masks
        ]
    
        not_empty_masks: List[NNCFTensor] = [mask for mask in input_masks if mask is not None]
        if not not_empty_masks:
            return None
    
        first_non_empty_mask = not_empty_masks[0]
    
        device = first_non_empty_mask.device
        filled_input_masks = []
        for i, mask in enumerate(input_masks):
            if mask is None:
                concat_axis = node.layer_attributes.axis if node.layer_attributes is not None else None  # type:ignore
>           if concat_axis is not None:
E           UnboundLocalError: local variable 'concat_axis' referenced before assignment

Please try to run pytorch and tensorflow pre-commit locally, as pre-commit works fine on develop branch you should be able to find bugs in your pr

@MaximProshin
Copy link
Collaborator

@RitikaxShakya , do you plan to continue to work on this PR?

@RitikaxShakya
Copy link
Author

@MaximProshin Hello! sorry for late reply, I was busy with my semester exams, so slowly working on issues, its almost completed, there are few more errors i need to fix.

@mlukasze
Copy link

hey @RitikaxShakya, will you have a time to finish it?

@RitikaxShakya
Copy link
Author

hey @RitikaxShakya, will you have a time to finish it?

Yes, i will try to finish it this week, sorry for being so late.

@mlukasze
Copy link

thanks! do not worry :)

@alexsu52
Copy link
Contributor

alexsu52 commented Aug 6, 2024

@RitikaxShakya please rebase your PR.

@RitikaxShakya
Copy link
Author

@RitikaxShakya please rebase your PR.

Thank you for informing, and i am trying to complete this issue soon

@MaximProshin
Copy link
Collaborator

@daniil-lyakhov @RitikaxShakya what is the status of this PR? Is it still active?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API Public API-impacting changes NNCF Common Pull request that updates NNCF Common
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants