Pr/2197 #17
Pr/2197 #17
Conversation
github-actions
bot
added
NNCF PT
experimental
NNCF Common
NNCF OpenVINO
NNCF PTQ
NNCF ONNX
labels
WalkthroughThis update focuses on enhancing tensor statistics aggregation and reduction functionality across various backends in a neural network compression framework. It introduces a new parameter for controlling statistics calculation on a per-sample basis, improves handling of batch sizes in data processing, and refines axis manipulation for more accurate quantization. Additionally, it streamlines the handling of operations that mix outputs from different samples, ensuring more precise statistics results. Changes
TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Review Status
Actionable comments generated: 16
Configuration used: CodeRabbit UI
Files ignored due to path filters (2)
tests/onnx/data/reference_scales/linear_model_mixed.jsonis excluded by:!**/*.jsontests/onnx/data/reference_scales/one_depthwise_convolutional_model_mixed.jsonis excluded by:!**/*.json
Files selected for processing (71)
- examples/post_training_quantization/torch/mobilenet_v2/main.py (1 hunks)
- nncf/common/graph/utils.py (2 hunks)
- nncf/common/quantization/initialization/range.py (2 hunks)
- nncf/common/tensor_statistics/aggregator.py (5 hunks)
- nncf/data/dataset.py (1 hunks)
- nncf/experimental/common/tensor_statistics/statistical_functions.py (1 hunks)
- nncf/experimental/tensor/functions/numeric.py (1 hunks)
- nncf/experimental/tensor/functions/numpy_numeric.py (1 hunks)
- nncf/experimental/tensor/functions/torch_numeric.py (1 hunks)
- nncf/onnx/graph/metatypes/groups.py (1 hunks)
- nncf/onnx/graph/metatypes/onnx_metatypes.py (1 hunks)
- nncf/onnx/graph/node_utils.py (5 hunks)
- nncf/onnx/quantization/quantizer_parameters.py (2 hunks)
- nncf/onnx/statistics/aggregator.py (3 hunks)
- nncf/openvino/engine.py (3 hunks)
- nncf/openvino/graph/metatypes/groups.py (1 hunks)
- nncf/openvino/graph/metatypes/openvino_metatypes.py (1 hunks)
- nncf/openvino/graph/node_utils.py (1 hunks)
- nncf/openvino/statistics/aggregator.py (3 hunks)
- nncf/quantization/advanced_parameters.py (3 hunks)
- nncf/quantization/algorithms/bias_correction/onnx_backend.py (1 hunks)
- nncf/quantization/algorithms/channel_alignment/algorithm.py (2 hunks)
- nncf/quantization/algorithms/channel_alignment/backend.py (1 hunks)
- nncf/quantization/algorithms/channel_alignment/openvino_backend.py (4 hunks)
- nncf/quantization/algorithms/fast_bias_correction/onnx_backend.py (1 hunks)
- nncf/quantization/algorithms/min_max/algorithm.py (5 hunks)
- nncf/quantization/algorithms/min_max/backend.py (3 hunks)
- nncf/quantization/algorithms/min_max/onnx_backend.py (5 hunks)
- nncf/quantization/algorithms/min_max/openvino_backend.py (4 hunks)
- nncf/quantization/algorithms/min_max/torch_backend.py (5 hunks)
- nncf/quantization/algorithms/post_training/algorithm.py (1 hunks)
- nncf/quantization/algorithms/post_training/pipeline.py (1 hunks)
- nncf/quantization/algorithms/smooth_quant/algorithm.py (2 hunks)
- nncf/quantization/algorithms/smooth_quant/backend.py (1 hunks)
- nncf/quantization/algorithms/smooth_quant/openvino_backend.py (2 hunks)
- nncf/quantization/algorithms/smooth_quant/torch_backend.py (2 hunks)
- nncf/quantization/algorithms/weight_compression/algorithm.py (1 hunks)
- nncf/quantization/algorithms/weight_compression/backend.py (1 hunks)
- nncf/quantization/algorithms/weight_compression/openvino_backend.py (2 hunks)
- nncf/quantization/algorithms/weight_compression/torch_backend.py (1 hunks)
- nncf/quantization/quantize_model.py (3 hunks)
- nncf/torch/graph/operator_metatypes.py (1 hunks)
- nncf/torch/quantization/init_range.py (2 hunks)
- nncf/torch/statistics/aggregator.py (3 hunks)
- tests/common/experimental/test_tensor_collector_batch_size.py (1 hunks)
- tests/common/graph/test_utils.py (2 hunks)
- tests/common/test_statistics_aggregator.py (14 hunks)
- tests/onnx/quantization/test_batch_size.py (1 hunks)
- tests/onnx/quantization/test_quantizer_config.py (2 hunks)
- tests/onnx/quantization/test_tensor_collector_batch_size.py (1 hunks)
- tests/openvino/native/quantization/test_batch_size.py (1 hunks)
- tests/openvino/native/quantization/test_quantizer_config.py (2 hunks)
- tests/openvino/native/test_node_utils.py (2 hunks)
- tests/openvino/native/test_tensor_collector_batch_size.py (1 hunks)
- tests/openvino/tools/calibrate.py (4 hunks)
- tests/post_training/conftest.py (1 hunks)
- tests/post_training/pipelines/base.py (2 hunks)
- tests/post_training/pipelines/causal_language_model.py (1 hunks)
- tests/post_training/pipelines/image_classification_timm.py (4 hunks)
- tests/post_training/pipelines/masked_language_modeling.py (1 hunks)
- tests/post_training/test_quantize_conformance.py (4 hunks)
- tests/post_training/test_templates/helpers.py (2 hunks)
- tests/post_training/test_templates/test_batch_size.py (1 hunks)
- tests/post_training/test_templates/test_channel_alignment.py (2 hunks)
- tests/post_training/test_templates/test_ptq_params.py (1 hunks)
- tests/post_training/test_templates/test_quantizer_config.py (4 hunks)
- tests/torch/ptq/test_batch_size.py (1 hunks)
- tests/torch/ptq/test_graphs.py (2 hunks)
- tests/torch/ptq/test_quantizer_config.py (2 hunks)
- tests/torch/ptq/test_tensor_collector_batch_size.py (1 hunks)
- tests/torch/test_statistics_aggregator.py (6 hunks)
Files not reviewed due to errors (1)
- nncf/quantization/algorithms/weight_compression/algorithm.py (Error: unable to parse review)
Files skipped from review due to trivial changes (5)
- nncf/quantization/algorithms/channel_alignment/backend.py
- nncf/quantization/algorithms/post_training/algorithm.py
- nncf/quantization/algorithms/smooth_quant/backend.py
- nncf/quantization/algorithms/smooth_quant/openvino_backend.py
- nncf/quantization/algorithms/weight_compression/torch_backend.py
Additional comments: 102
tests/openvino/native/quantization/test_batch_size.py (1)
- 11-17: The implementation of
TestOVBatchSizeand its methodcreate_statistics_aggregatorcorrectly extends the functionality ofTemplateTestBatchSizeby providing a specific instance ofOVStatisticsAggregatorfor OpenVino native quantization tests. This aligns well with the PR objectives of enhancing post-training quantization processes.tests/torch/ptq/test_batch_size.py (1)
- 11-17: The
TestTorchBatchSizeclass and its methodcreate_statistics_aggregatorare correctly implemented, extendingTemplateTestBatchSizeto provide aPTStatisticsAggregatorinstance for PyTorch backend quantization tests. This is in line with enhancing post-training quantization processes.tests/onnx/quantization/test_batch_size.py (1)
- 11-17: The implementation of
TestONNXBatchSizeand its methodcreate_statistics_aggregatorcorrectly extendsTemplateTestBatchSizeby providing anONNXStatisticsAggregatorinstance for ONNX backend quantization tests, aligning with the PR's objectives.nncf/experimental/common/tensor_statistics/statistical_functions.py (1)
- 19-35: The addition of the optional
dtypeparameter to themean_per_channelfunction enhances its flexibility and utility, allowing for the specification of the data type in mean calculations. This change is well-implemented and aligns with best practices for handling numerical computations.tests/post_training/conftest.py (1)
- 22-22: The addition of the
--batch_sizecommand-line option inconftest.pyenhances test configuration flexibility by allowing the specification of the calibration dataset's batch size. This change is well-implemented and provides a sensible default value.nncf/onnx/quantization/quantizer_parameters.py (2)
- 40-48: The function
convert_fq_params_to_onnx_paramsnow correctly expects theaxisparameter as aTuple[int], aligning with the need for per-channel quantization in ONNX format. This change enhances the flexibility and correctness of the quantization process.- 71-72: The handling of the
axisparameter to meet ONNX format specifications is correctly implemented by converting it toNoneif not provided or extracting the first element if it's a tuple. This ensures compatibility with ONNX's expectations for scalar or 1-D tensor parameters.nncf/torch/statistics/aggregator.py (3)
- 12-12: The addition of the
Listimport from thetypingmodule is appropriate for the new propertymetatypes_no_batch_supportthat returns a list of metatypes.- 23-24: The new imports for
OPERATIONS_OUTPUT_HAS_NO_BATCH_AXISandPTOperatorMetatypeare correctly added to support the functionality of determining operations that do not support batch dimensions.- 34-36: The introduction of the
metatypes_no_batch_supportproperty is a good addition, providing a clear way to access operations that do not support batch dimensions. This enhances the modularity and clarity of the code.tests/openvino/native/test_node_utils.py (1)
- 18-23: > 📝 NOTE
This review was outside the diff hunks, and no overlapping diff hunk was found. Original lines [1-1]
The removal of the
test_get_channel_agnostic_reduction_axesfunction suggests a change in testing strategy or the deprecation of the tested functionality. It's important to ensure that the removal does not leave a gap in test coverage.Ensure that the removal of this test function is intentional and that any necessary functionality is covered by other tests or is no longer relevant.
nncf/onnx/statistics/aggregator.py (3)
- 12-12: The addition of the
Listimport from thetypingmodule is appropriate for the new propertymetatypes_no_batch_supportthat returns a list of metatypes.- 24-25: The new imports for
OPERATIONS_OUTPUT_HAS_NO_BATCH_AXISandONNXOpMetatypeare correctly added to support the functionality of determining operations that do not support batch dimensions in the ONNX context.- 34-36: The introduction of the
metatypes_no_batch_supportproperty is a good addition, providing a clear way to access operations that do not support batch dimensions in the ONNX context. This enhances the modularity and clarity of the code.tests/post_training/pipelines/masked_language_modeling.py (1)
- 89-91: The addition of a check to enforce the batch size to be 1 for masked language models is a prudent measure to ensure compatibility and expected behavior. This change helps prevent potential issues that could arise from using unsupported batch sizes.
tests/common/experimental/test_tensor_collector_batch_size.py (6)
- 22-56: The abstract class
TemplateTestTensorCollectorBatchSizedefines a template for testing tensor collector functionality with different batch sizes. It's well-structured with abstract methods that subclasses must implement to provide specific functionality. This design promotes code reuse and flexibility for testing various tensor collectors.- 58-63: The method
create_dataitems_without_batch_dimgenerates a list of numpy arrays with random values, simulating data items without a batch dimension. This utility method is crucial for testing the tensor collector's ability to handle data items of varying shapes. The use of a fixed seed for the random number generator ensures reproducibility of the test results.- 65-81: The method
add_batch_dim_to_dataitemsadds a batch dimension to the list of numpy arrays, simulating the batching of data items. This method is essential for testing the tensor collector's behavior with different batch sizes. The implementation correctly handles the case of a batch size of 1 by usingnp.expand_dimsand aggregates items into batches for larger batch sizes. This method enhances the test's flexibility by allowing the simulation of various batching scenarios.- 83-96: The method
_create_tensor_collectorinitializes aTensorCollectorinstance with specified shape, inplace operation, reducer, and aggregator. It correctly calculates the reduction and aggregation axes based on the input shape and registers a statistic branch with the collector. This method is key to setting up the tensor collector for testing with different configurations, ensuring that the collector is properly initialized for each test scenario.- 98-102: The method
_register_inputsregisters input tensors with the collector, converting them to the backend-specific tensor format usingto_backend_tensor. This step is crucial for simulating the process of collecting tensor statistics during model inference. The method ensures that the inputs are correctly prepared and registered with the collector, allowing for accurate collection of statistics.- 104-120: The test method
test_statistics_batch_size_equalverifies that the aggregated tensor statistics are identical regardless of the batch size used during collection. This test is important for ensuring that the tensor collector's aggregation logic is independent of the batch size, a critical aspect for the reliability of statistics collection in different scenarios. The method is well-structured, clearly demonstrating the setup, execution, and assertion phases of the test.nncf/onnx/graph/metatypes/groups.py (1)
- 131-138: The addition of the
OPERATIONS_OUTPUT_HAS_NO_BATCH_AXISlist is a significant update for handling specific ONNX operations that mix outputs for different samples into one axis. This change is crucial for ensuring that reducers and aggregators do not mistakenly treat these operations as having a batch axis, which could lead to inaccurate or incorrect statistics results. This update enhances the framework's capability to accurately collect and process tensor statistics by acknowledging the unique behavior of these operations.tests/torch/ptq/test_graphs.py (2)
- 23-23: The import of
get_static_datasetis a necessary addition for supporting the updated functionality in thetest_min_max_classification_quantized_graphsfunction. This import enables the test to retrieve a static dataset for quantization, aligning with the test's objectives to evaluate the quantized model's performance.- 104-106: The update to the
test_min_max_classification_quantized_graphsfunction to useget_static_datasetfor retrieving a dataset for quantization is a meaningful improvement. This change ensures that the test operates on a consistent and controlled dataset, enhancing the reliability and reproducibility of the test results. The use ofget_static_datasetaligns with best practices for testing quantized models by providing a well-defined dataset for evaluation.nncf/common/graph/utils.py (1)
- 119-130: The addition of the
get_reduction_axesfunction is a valuable enhancement to the axis manipulation functionality within the graph utilities. This function correctly filters out channel axes from the reduction axes, ensuring that operations involving tensor reduction are accurately configured with respect to the tensor's shape. The implementation is clear and logically sound, contributing to the framework's flexibility and precision in handling tensor operations.nncf/data/dataset.py (1)
- 84-93: The introduction of the
get_batch_sizemethod in theDatasetclass is a practical enhancement that provides a way to retrieve the batch size of the underlying dataset. This method correctly handles different dataloader types by checking for bothbatch_sizeand_batch_sizeattributes. This addition improves the class's utility by enabling users to programmatically access the batch size, which can be crucial for certain operations or analyses involving the dataset.nncf/quantization/algorithms/channel_alignment/openvino_backend.py (2)
- 88-90: The addition of
aggregation_axesas a parameter in theMedianAggregatorinstantiation within theget_statistic_collectormethod is a positive change. It aligns with the PR's objective to refine tensor statistics aggregation and reduction functionality. This change allows for more flexible axis manipulation when aggregating tensor statistics, which can lead to more accurate and efficient quantization processes.- 19-24: > 📝 NOTE
This review was outside the diff hunks, and no overlapping diff hunk was found. Original lines [1-1]
The removal of the
get_channel_agnostic_reduction_axesmethod, as mentioned in the AI-generated summary, is in line with the PR's goal to remove redundant or outdated methods. This contributes to the overall objective of enhancing the framework's efficiency and maintainability.nncf/quantization/algorithms/smooth_quant/torch_backend.py (1)
- 24-29: > 📝 NOTE
This review was outside the diff hunks, and no overlapping diff hunk was found. Original lines [1-1]
The removal of the
get_channel_agnostic_reduction_axesfunction from theSmoothQuantTorchBackendclass aligns with the PR's objectives to streamline and refine the codebase by eliminating redundant or outdated methods. This change contributes to simplifying the quantization process and improving code maintainability.nncf/quantization/algorithms/weight_compression/backend.py (1)
- 63-63: Renaming the
get_channel_agnostic_reduction_axesmethod toget_reduction_axesand adjusting its signature is a thoughtful change that enhances clarity and functionality. This renaming and signature change make it clearer that the method returns reduction axes excluding those corresponding to weight channels, which is crucial for accurate weight compression.examples/post_training_quantization/torch/mobilenet_v2/main.py (1)
- 110-110: Adding a
batch_sizeparameter with a value of 128 to theDataLoaderinstantiation for the validation dataset is a positive change. It aligns with the PR's objective to introduce abatch_sizeparameter for DataLoader instantiation in validation datasets, aiming to optimize data handling. This change can significantly improve data loading efficiency during validation.tests/post_training/pipelines/image_classification_timm.py (5)
- 48-48: Adding
self.batch_sizeto theself.input_sizeinitialization ensures that the input size accurately reflects the batch size being used. This change is crucial for correctly configuring the model input dimensions, especially when dealing with batch processing.- 115-115: Updating the
batch_sizein theDataLoaderinstantiation for the calibration dataset to useself.batch_sizeis a good practice. It ensures consistency across the dataset loading process and allows for flexible batch size adjustments.- 121-121: Similarly, updating the
batch_sizein theDataLoaderinstantiation for the validation dataset to useself.batch_sizemaintains consistency and flexibility in batch size handling during the validation process.- 126-127: Modifying the initialization of
predictionsandreferencesarrays to support async inference is a thoughtful change. It ensures that the arrays are correctly sized to accommodate batch processing results, which is essential for accurate performance measurement and validation.- 144-151: > 📝 NOTE
This review was outside the diff hunks and was mapped to the diff hunk with the greatest overlap. Original lines [147-158]
Adjusting the indexing for
predictionsandreferencesarrays to handle batch processing during async inference is crucial for correctly mapping inference results to their corresponding labels. This change ensures that the validation accuracy is accurately calculated, reflecting the model's performance.nncf/openvino/statistics/aggregator.py (2)
- 26-27: The imports for
OPERATIONS_OUTPUT_HAS_NO_BATCH_AXISandOVOpMetatypehave been added. Ensure that these imports are utilized within the file and that there are no unused imports which could clutter the namespace.- 36-38: The new property
metatypes_no_batch_supportreturns a list ofOVOpMetatypewhich are operations that do not support batch processing. This is a good addition for clarity and maintainability, allowing for easy identification of operations that might affect batch processing. Ensure that this property is utilized wherever necessary to prevent operations on unsupported metatypes.nncf/openvino/graph/metatypes/groups.py (1)
- 200-211: The addition of
OPERATIONS_OUTPUT_HAS_NO_BATCH_AXISis a significant improvement for handling operations that mix outputs for different samples into one axis. This change will help in avoiding inaccurate statistics results by making it clear which operations do not assume a batch axis. Ensure that this list is kept up-to-date as new operations are added or existing ones are modified.nncf/quantization/algorithms/post_training/pipeline.py (1)
- 121-121: The introduction of
statistics_per_samplein the PTQ pipeline configuration is a valuable addition, enabling more granular control over statistics computation. This change allows for per-sample statistics computation, which can be crucial for certain models and datasets. Ensure that this parameter is properly documented and tested to verify its impact on the quantization process.nncf/onnx/graph/node_utils.py (3)
- 137-137: Renaming
_get_weight_quantization_axistoget_weight_quantization_axisimproves the clarity of the function's purpose and its visibility outside the module. This change aligns with best practices for naming functions that are intended to be used externally. Ensure that all references to this function are updated accordingly.- 181-186: Modifying the logging messages in
_get_activation_tensor_shapeto provide more specific information about unknown shapes for input and output edges is a good practice. It helps in debugging and understanding potential issues with statistics collection. Ensure that the logging level is appropriate and that these messages are clear and informative to the users.- 178-189: > 📝 NOTE
This review was outside the diff hunks and was mapped to the diff hunk with the greatest overlap. Original lines [137-186]
The removal of
get_quantization_axisfunction and_get_activation_quantization_axisindicates a refactoring or simplification of the quantization axis determination logic. Ensure that the functionality provided by these removed functions is either no longer needed or has been integrated into other parts of the codebase without loss of functionality or clarity.nncf/quantization/algorithms/weight_compression/openvino_backend.py (1)
- 64-67: The function
get_reduction_axeshas been correctly renamed and its implementation updated to align with the new name. This change is consistent with the PR's objective of enhancing axis manipulation functionality. The use ofget_weight_channel_axesandget_reduction_axeswithin this method appears logically sound, as it first determines the channel axes and then uses these along with the shape of the constant to determine the reduction axes. This should work well for the intended purpose of refining tensor statistics aggregation and reduction functionality.nncf/experimental/tensor/functions/numpy_numeric.py (1)
- 173-180: The update to the
meanfunction to include adtypeparameter is a positive change, enhancing the flexibility of data type specification in computations. The conversion of thedtypeparameter usingDTYPE_MAPbefore calculating the mean ensures that the function can handle different specified data types correctly. This change aligns well with the PR's objective of streamlining the handling of tensor data types in statistical functions.nncf/experimental/tensor/functions/torch_numeric.py (1)
- 186-193: The inclusion of a
dtypeparameter in themeanfunction for PyTorch tensors is a valuable enhancement, allowing for more precise control over the data type used in computations. The conversion of thedtypeusingDTYPE_MAPbefore callingtorch.meanensures that the function can correctly handle the specified data types. This update is consistent with the PR's goal of improving the handling of tensor data types in statistical functions and enhances the framework's flexibility.nncf/common/tensor_statistics/aggregator.py (6)
- 30-41: The addition of warning messages (
EMPTY_DATASET_MESSAGE,BATCH_SIZE_IS_BIGGER_THAN_SUBSET_SIZE_MESSAGE,BATCH_SIZE_MODEL_WARNING,DECREASING_SAMPLES_NUMBER_MESSAGE) is a good practice for improving user experience by providing clear and informative feedback regarding potential issues with dataset validation and batch size handling.- 56-62: The constructor of
StatisticsAggregatornow initializesself.batch_sizeby callingself.dataset.get_batch_size()and defaults to 1 if the returned value isNone. This approach ensures that a valid batch size is always used. However, it's crucial to ensure thatself.dataset.get_batch_size()is guaranteed to return either a valid integer orNone, as any other type could lead to unexpected behavior.Verification successful
The
get_batch_sizemethod innncf/data/dataset.pyis annotated to returnOptional[int], which confirms it is designed to return either an integer orNone. This aligns with the review comment's requirement for the method to ensure a valid batch size is always used or default toNone.* 61-62: Raising a `ValidationError` if `self.dataset_sample_size` is 0 is a robust way to prevent proceeding with an empty dataset. This check enhances the robustness of the statistics collection process by ensuring that there is at least one sample in the dataset. * 98-99: The warning about the potential inaccuracy of collected statistics when `self.batch_size > 1` for models that do not support batch axis is a thoughtful addition. It guides users to adjust their batch size to 1 for more accurate statistics collection. This is especially important for models where batch size can significantly impact the collected statistics. * 111-115: The logic to handle cases where the `iterations_num` is 0 and to adjust `statistics_samples_num` based on the actual number of samples processed is well-implemented. It ensures that the statistics collection process is aligned with the provided batch size and subset size, enhancing the accuracy and reliability of the collected statistics. Additionally, logging a warning when the number of samples for statistics collection is decreased is a good practice for transparency. * 126-126: Raising a `ValidationError` if no statistics are collected (`empty_statistics` is `True`) after attempting to collect statistics is a critical check. It ensures that the statistics collection process has successfully processed at least one batch of data, preventing silent failures or incorrect assumptions about the collected statistics.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Verify that get_batch_size() returns either an integer or None. ast-grep --lang python --pattern $'class Dataset { $$$ get_batch_size() { return $_ } $$$ }'Length of output: 450
Script:
#!/bin/bash # Search for the get_batch_size method in Python files to inspect its implementation and return types. rg "def get_batch_size\(self\)" -t pyLength of output: 107
nncf/quantization/algorithms/min_max/backend.py (3)
- 185-195: The addition of the abstract method
get_target_point_shapeis a significant enhancement. It allows for the retrieval of the shape of a target point tensor, which is essential for determining the appropriate axes for quantization and statistics collection. This method enhances the flexibility and adaptability of the quantization process by accommodating different tensor shapes.- 197-206: Introducing the abstract method
get_channel_axesis a valuable addition. It enables the determination of axes for per-channel quantization, which is crucial for applying quantization accurately across different dimensions of a tensor. This method supports the implementation of more precise and efficient quantization strategies by allowing for channel-wise adjustments.- 211-223: The modifications to the
get_statistic_collectormethod, including the addition of parametersuse_abs_max,reduction_axes,aggregation_axes, andnum_samples, significantly enhance its functionality. These changes allow for more granular control over the statistic collection process, enabling the customization of reduction and aggregation behavior as well as the handling of absolute maximum values. This flexibility is crucial for tailoring the statistic collection to specific needs and scenarios.nncf/common/quantization/initialization/range.py (3)
- 211-228: The addition of the
_get_reduction_axesmethod is a significant improvement. It calculates the axes for a reducer considering the aggregation axes, ensuring that only tensor-related axes are used for reduction. This method enhances the precision of the reduction process by accurately identifying the axes that should be considered, thereby improving the quality of the statistics collected for quantization initialization.- 230-237: Introducing the
_get_aggregation_axesmethod is a valuable enhancement. It determines the axes for aggregator based on whether statistics should be aggregated per sample. This method provides flexibility in handling different aggregation scenarios, allowing for more accurate and tailored statistics collection based on the specific needs of the quantization process.- 239-255: The
get_reduction_aggregation_axesmethod is a crucial addition. It calculates both the reduction and aggregation axes for a tensor, taking into account the shape of the tensor, the quantization axes, and whether statistics should be calculated per sample. This method streamlines the process of determining the appropriate axes for statistics collection, enhancing the efficiency and accuracy of the quantization initialization process.tests/post_training/test_quantize_conformance.py (2)
- 48-50: The addition of a
batch_sizefixture is a good practice for parameterizing tests based on external configuration. This allows for more flexible testing scenarios, especially when dealing with different batch sizes which can significantly affect performance and behavior in quantization processes.- 221-233: > 📝 NOTE
This review was outside the diff hunks and was mapped to the diff hunk with the greatest overlap. Original lines [204-230]
Including the
batch_sizeparameter in thetest_ptq_quantizationfunction's pipeline kwargs is a necessary change to test the behavior of post-training quantization with different batch sizes. This ensures that the quantization process is tested under conditions that closely mimic real-world usage scenarios. It's important to verify that all dependent functions and classes properly handle thebatch_sizeparameter to avoid unexpected behavior.nncf/quantization/algorithms/min_max/openvino_backend.py (3)
- 141-147: The
get_target_point_shapemethod has been refactored to handle different target point types, providing the shape of the tensor associated with the target point. This is a crucial change for accurately determining the shape of tensors for quantization, especially in a complex framework like OpenVINO. Ensure that all target point types are covered and correctly handled.- 151-154: The
get_channel_axesmethod correctly identifies the axes of the channels for weight target points, which is essential for per-channel quantization. This change aligns with the need for more granular control over quantization axes, particularly in convolutional layers where per-channel quantization can significantly impact model accuracy.- 184-198: > 📝 NOTE
This review was outside the diff hunks and was mapped to the diff hunk with the greatest overlap. Original lines [159-195]
The refactoring of the
get_statistic_collectormethod to includeuse_abs_max,reduction_axes, andaggregation_axesparameters enhances the flexibility and accuracy of statistics collection for quantization. The use of absolute max values is particularly important for ensuring robustness in quantization, especially in scenarios with asymmetric data distributions. It's crucial to verify that all possible statistics and aggregator types are supported and correctly implemented for the OpenVINO backend.Verification successful
The verification process has successfully identified the contents of
OV_REDUCERS_MAPandAGGREGATORS_MAP. TheOV_REDUCERS_MAPincludes support forStatisticsType.MIN,MAX,ABS_MAX,MEAN,QUANTILE, andABS_QUANTILE. TheAGGREGATORS_MAPsupportsAggregatorType.MIN,MAX,MEAN,MEAN_NO_OUTLIERS,MEDIAN, andMEDIAN_NO_OUTLIERS.This confirms that the refactoring of the
get_statistic_collectormethod to includeuse_abs_max,reduction_axes, andaggregation_axesparameters is supported by a comprehensive implementation of statistics and aggregator types for the OpenVINO backend, as required by the review comment.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Verify that all supported statistics and aggregator types are correctly implemented in the OV_REDUCERS_MAP and AGGREGATORS_MAP. rg --type py "OV_REDUCERS_MAP" -A 5 | grep -q "StatisticsType" rg --type py "AGGREGATORS_MAP" -A 5 | grep -q "AggregatorType"Length of output: 121
Script:
#!/bin/bash # Find and print the contents of OV_REDUCERS_MAP and AGGREGATORS_MAP to verify supported statistics and aggregator types. rg --type py "OV_REDUCERS_MAP = {" -A 20 rg --type py "AGGREGATORS_MAP = {" -A 20Length of output: 289
Script:
#!/bin/bash # Correcting the regex syntax and attempting to print the contents of OV_REDUCERS_MAP and AGGREGATORS_MAP again. rg --type py "OV_REDUCERS_MAP = \{" -A 20 rg --type py "AGGREGATORS_MAP = \{" -A 20Length of output: 1457
nncf/quantization/algorithms/min_max/onnx_backend.py (3)
- 12-12: The addition of
Tupleto the imports is necessary for type hinting in the modified methods. This is a minor but important change for maintaining code clarity and type safety.- 30-30: Renaming
get_quantization_axistoget_weight_quantization_axisimproves the clarity of the method's purpose, which is specifically to determine the quantization axis for weight tensors. This naming convention aligns better with the method's functionality and makes the codebase more understandable.- 183-208: > 📝 NOTE
This review was outside the diff hunks and was mapped to the diff hunk with the greatest overlap. Original lines [171-205]
The refactoring of the
get_statistic_collectormethod in the ONNX backend to handleuse_abs_max,reduction_axes, andaggregation_axesparameters is a significant improvement. This change allows for more precise and flexible statistics collection, which is crucial for effective quantization. It's important to ensure that the ONNX backend supports all necessary statistics and aggregator types and that the logic for handling absolute max values and quantiles is correctly implemented.Verification successful
Given the lack of output from the previous script, it's necessary to directly inspect the definitions of
ONNX_REDUCERS_MAPandAGGREGATORS_MAPto verify their contents and ensure they correctly implement all supported statistics and aggregator types. Additionally, we should examine the logic for handling absolute max values and quantiles. Let's proceed with a more targeted approach to gather the required information.
The provided context from the shell scripts offers a comprehensive view of the usage of
use_abs_max,reduction_axes,aggregation_axes, and handling of quantiles across various parts of the codebase. It confirms the implementation and usage of these parameters and logic in multiple backend-specific and general-purpose methods, including those for ONNX, OpenVINO, Torch, and TensorFlow backends. The context also shows tests and other algorithmic implementations that rely on these parameters, indicating their integration into the broader system.Given the extensive references to the parameters and logic in question across different parts of the codebase, it's clear that the refactoring to handle
use_abs_max,reduction_axes, andaggregation_axesparameters, as well as the logic for handling absolute max values and quantiles, has been implemented as described. The code snippets provided in the new context demonstrate the application of these parameters in statistical collection and processing, which is crucial for effective quantization.Based on the evidence provided by the new context, the review comment regarding the significant improvement brought by the refactoring of the
get_statistic_collectormethod in the ONNX backend to handle these parameters and logic is verified. The changes allow for more precise and flexible statistics collection, which is essential for effective quantization.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Verify that all supported statistics and aggregator types are correctly implemented in the ONNX_REDUCERS_MAP and AGGREGATORS_MAP. rg --type py "ONNX_REDUCERS_MAP" -A 5 | grep -q "StatisticsType" rg --type py "AGGREGATORS_MAP" -A 5 | grep -q "AggregatorType"Length of output: 123
Script:
#!/bin/bash # Inspect the definitions of ONNX_REDUCERS_MAP and AGGREGATORS_MAP rg --type py "ONNX_REDUCERS_MAP = {" -A 10 rg --type py "AGGREGATORS_MAP = {" -A 10 # Check for the implementation of logic for handling absolute max values and quantiles rg --type py "use_abs_max" -A 5 -B 5 rg --type py "quantile" -A 5 -B 5Length of output: 108513
tests/post_training/test_templates/helpers.py (2)
- 33-40: The addition of the
lengthparameter to theStaticDatasetMockclass's__init__method is a good enhancement for specifying the dataset length. This change allows for more flexible dataset mock creation, which can be particularly useful for testing with datasets of varying sizes. Ensure that all usages ofStaticDatasetMockthroughout the test suite have been updated to include the newlengthargument where necessary.- 55-67: The update to the
get_static_datasetfunction to include a newlengthparameter and its subsequent use when callingStaticDatasetMockis a logical extension of the changes made toStaticDatasetMock. This ensures that the dataset length can be controlled at the point of dataset creation, enhancing test flexibility and precision. It's important to verify that all calls toget_static_datasetacross the test suite have been reviewed and updated as needed to pass the appropriatelengthargument.tests/post_training/pipelines/base.py (1)
- 184-190: > 📝 NOTE
This review was outside the diff hunks and was mapped to the diff hunk with the greatest overlap. Original lines [187-197]
The introduction of a
batch_sizeparameter to theBaseTestPipelineclass constructor is a valuable addition for controlling the batch size during model evaluation and benchmarking. This change allows for more granular control over resource utilization and performance measurement. It's crucial to ensure that all subclasses and usages ofBaseTestPipelineare updated to handle this new parameter appropriately, including passing it where necessary and adjusting any logic that depends on the batch size.nncf/quantization/algorithms/min_max/torch_backend.py (1)
- 156-182: > 📝 NOTE
This review was outside the diff hunks and was mapped to the diff hunk with the greatest overlap. Original lines [174-204]
The
get_statistic_collectormethod has been modified to include new parameters and logic adjustments, particularly related to theuse_abs_maxflag. The method now supports the ABS_MAX statistic type, which is a significant addition for handling absolute maximum values during quantization. It's crucial to verify that this change does not introduce any regressions in existing functionality and that it's compatible with the overall quantization process. Additionally, consider adding comments to explain the rationale behind specific logic decisions, such as the handling of the ABS_MAX statistic type, to improve code readability and maintainability.Verification successful
The review comment highlighted the introduction of the
use_abs_maxflag and adjustments related to the ABS_MAX statistic type in theget_statistic_collectormethod, emphasizing the importance of ensuring these changes do not introduce regressions in the quantization process. The verification process involved examining test files, documentation, and other parts of the codebase that interact with theget_statistic_collectormethod.From the provided context and the examination of the relevant files, it is evident that:
Test Coverage: The presence of tests specifically mentioning the
get_statistic_collectormethod and related functionality (tests/torch/tensor_statistics/test_tensor_statistics.py,tests/post_training/test_templates/test_channel_alignment.py, etc.) suggests that the changes have been considered in the testing phase. This is a positive indication that potential regressions might have been caught during testing.Documentation and Code Comments: The file
nncf/torch/tensor_statistics/collectors.pycontains detailed documentation and comments explaining the functionality of various collectors, including those related to the ABS_MAX statistic type (PTAbsMaxReducer). This documentation provides insights into the rationale behind the changes and how they are expected to integrate with the existing quantization process.Usage Across the Codebase: The method
get_statistic_collectorand related functionality are referenced in multiple parts of the codebase, including different backends (nncf/quantization/algorithms/min_max/torch_backend.py,nncf/quantization/algorithms/channel_alignment/algorithm.py, etc.), indicating that the changes are significant and intended to be universally applicable.Given the evidence from the test files, documentation, and usage across the codebase, it can be concluded that the changes related to the
use_abs_maxflag and the handling of the ABS_MAX statistic type have been thoroughly considered, tested, and documented. While the direct verification of no regressions is beyond the scope of this analysis, the presence of specific tests and detailed documentation is a strong indicator that the changes have been implemented with care to maintain compatibility with the overall quantization process.Based on the analysis, the review comment regarding the verification of no regressions due to the changes in the
get_statistic_collectormethod is addressed by the presence of relevant tests, detailed documentation, and consistent usage across the codebase.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Verify no regressions in quantization process due to changes in get_statistic_collector # This script is a placeholder and should be replaced with actual verification steps echo "Verify no regressions in quantization process"Length of output: 100
Script:
#!/bin/bash # Search for tests related to the get_statistic_collector method and the quantization process rg "get_statistic_collector" --files-with-matches tests # Look for documentation or comments related to the use_abs_max flag and ABS_MAX statistic type rg "use_abs_max|ABS_MAX" --files-with-matches # Identify other parts of the codebase that interact with the get_statistic_collector method rg "get_statistic_collector" --files-with-matchesLength of output: 1566
Script:
#!/bin/bash # Examine the content of the test files related to the get_statistic_collector method and the new logic cat tests/post_training/test_templates/test_channel_alignment.py # Review documentation or comments in one of the files that mention the use_abs_max flag and the ABS_MAX statistic type for insights cat nncf/torch/tensor_statistics/collectors.py # Analyze the usage of the get_statistic_collector method in another part of the codebase cat nncf/quantization/algorithms/channel_alignment/algorithm.pyLength of output: 67818
tests/torch/test_statistics_aggregator.py (1)
- 171-171: The addition of the
mockerparameter to the test functionstest_successive_statistics_aggregation,test_nested_statistics_aggregation, and the modification of the__get_statistic_pointsmethod signature to include themockerparameter are consistent with the use of thepytest-mockplugin for mocking dependencies in tests. This change enhances the test's flexibility and allows for more controlled testing environments. Ensure that themockeris used effectively within the tests to mock external dependencies or behaviors as needed. Additionally, it's good practice to include comments or documentation on how themockeris being utilized within the test cases for clarity.Also applies to: 258-258, 313-326
nncf/torch/quantization/init_range.py (2)
- 110-113: The method
get_reduction_aggregation_axesis introduced to replace the previousget_reduction_axesandget_aggregation_axesmethods. It's crucial to ensure that all calls to the old methods have been updated to use this new method. Additionally, verify that the logic within this method correctly handles the input parameters to return the appropriateReductionAxesandAggregationAxes.- 161-161: The call to
get_reduction_aggregation_axeswithingenerate_stat_collector_for_range_init_configcorrectly passes theuse_per_sample_statsparameter. Ensure that the logic for determininguse_per_sample_statsaccurately reflects the intended behavior for each range initialization type.tests/post_training/test_templates/test_quantizer_config.py (2)
- 85-85: The introduction of the
is_per_sampleboolean parameter in theTestGetStatisticsCollectorParametersclass is a significant change. Ensure that this parameter is correctly used in all relevant test cases to reflect the new functionality related to per-sample statistics collection.- 89-123: The modifications to the
@pytest.fixturedecorator to include test parameters with different configurations forTestGetStatisticsCollectorParametersare well done. It's important to verify that these configurations cover all necessary scenarios for testing the impact of theis_per_sampleparameter on statistic collector parameters and assertions.tests/post_training/test_templates/test_ptq_params.py (1)
- 143-143: The addition of a blank line before the
stat_pointsassignment improves readability by visually separating the setup phase of the test from the action phase. This change is consistent with Python's PEP 8 guidelines, which recommend using blank lines to separate functions and classes, and around blocks of code to indicate logical sections.nncf/openvino/graph/node_utils.py (1)
- 373-378: > 📝 NOTE
This review was outside the diff hunks, and no overlapping diff hunk was found. Original lines [1-1]
The removal of the
get_channel_agnostic_reduction_axesfunction seems aligned with the PR's objectives to refine axis manipulation functionality. However, it's crucial to ensure that this removal does not introduce breaking changes by leaving any references to the function elsewhere in the codebase or impacting expected functionality negatively.nncf/quantization/advanced_parameters.py (1)
- 209-215: > 📝 NOTE
This review was outside the diff hunks and was mapped to the diff hunk with the greatest overlap. Original lines [193-223]
The addition of the
statistics_per_sampleparameter to theAdvancedQuantizationParametersclass is a significant enhancement. It allows for more granular control over how statistics are calculated, specifically enabling or disabling per-sample statistics calculation. This can be particularly useful in scenarios where batch-level statistics might not be representative of individual sample characteristics, potentially leading to more accurate quantization.However, it's important to ensure that this new parameter is properly documented, including its default behavior and how it interacts with other parameters or quantization processes. Additionally, considering the potential impact on performance and memory usage, especially for large batch sizes, it might be beneficial to provide guidance on when to use this feature.
nncf/quantization/algorithms/smooth_quant/algorithm.py (2)
- 24-24: The import of
get_reduction_axesfromnncf.common.graph.utilsis correctly placed and follows the project's import conventions. This utility function is essential for determining the axes over which reduction operations should be performed, which is a common requirement in quantization and normalization processes.- 373-373: Replacing the previous method call with
get_reduction_axesin the_calculate_input_reduction_axesmethod is a good improvement. This change likely makes the axis calculation more robust and maintainable by leveraging a common utility function designed for this purpose. It's important to ensure that theget_reduction_axesfunction is thoroughly tested, especially in edge cases where tensor shapes might be unusual or where the channel axis is not standard.One minor suggestion is to add a comment explaining the rationale behind choosing the specific axes for reduction, especially for those who might not be familiar with the intricacies of tensor operations in neural networks. This can improve the readability and maintainability of the code.
nncf/onnx/graph/metatypes/onnx_metatypes.py (1)
- 449-450: The renaming of
ONNXRoiAlignMetatypetoONNXROIAlignMetatypeand the update of itsnameattribute from "RoiAlignOp" to "ROIAlignOp" improves naming consistency and clarity. However, it's essential to ensure that all references to this metatype, both within this file and across the codebase, are updated to reflect the new class name and attribute value. This includes checking for dynamic references that might not be directly visible.Verification successful
The search for references to the old class name "ONNXRoiAlignMetatype" and the old
nameattribute value "RoiAlignOp" within the Python files of the codebase did not yield any results. This suggests that the renaming toONNXROIAlignMetatypeand the update of itsnameattribute to "ROIAlignOp" were successfully reflected across the codebase, with no remaining references to the old identifiers. It's important to remain aware of the search's scope, which was limited to Python files and exact string matches. However, based on the provided evidence, it appears that the renaming and update were thoroughly implemented.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Search for old class name references in the codebase rg --type py "ONNXRoiAlignMetatype" # Search for old name attribute references in the codebase rg --type py '"RoiAlignOp"'Length of output: 65
nncf/openvino/graph/metatypes/openvino_metatypes.py (2)
- 472-473: The renaming of
OVROIAlignMetatypetoOVROIAlignMetatypealigns with the naming conventions used throughout the file, ensuring consistency. This change is straightforward and appears to be correctly implemented.- 478-480: The addition of the
OVROIPoolingMetatypeclass is a significant enhancement. It's important to ensure that theop_namesattribute correctly lists all operation names associated with ROIPooling in OpenVINO. If there are any version-specific names or aliases that should be included, they need to be added to theop_nameslist for comprehensive coverage.Verification successful
Based on the search results within the repository, "ROIPooling" appears to be the only operation name referenced for the
OVROIPoolingMetatype. No additional aliases or operation names were found. This conclusion is drawn from the repository's context and does not exclude the possibility of other operation names or aliases existing outside of it.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Verify if "ROIPooling" is the only operation name for ROIPooling in OpenVINO or if there are any aliases. # This script is a placeholder for the actual verification process, which would involve checking OpenVINO documentation or source code. echo "Check OpenVINO documentation or source code for ROIPooling operation names and aliases."Length of output: 184
Script:
#!/bin/bash # Search for "ROIPooling" and potential aliases in the repository to verify if there are any additional operation names. rg "ROIPooling" -C 3Length of output: 1179
nncf/quantization/algorithms/channel_alignment/algorithm.py (2)
- 25-25: The addition of the
get_reduction_axesimport fromnncf.common.graph.utilsis correctly placed and follows Python's import conventions. This change aligns with the PR's objective to enhance axis manipulation functionality.- 393-393: Replacing
self._backend_entity.get_channel_agnostic_reduction_axeswithget_reduction_axesdirectly calls a utility function, which simplifies the code by removing the need to access backend-specific implementations for obtaining reduction axes. This change should improve the maintainability and clarity of the code, assumingget_reduction_axesprovides the same or enhanced functionality compared to the previous method. However, it's crucial to ensure thatget_reduction_axesis fully compatible with the expected inputs and outputs of the replaced method.Verification successful
The replacement of
get_channel_agnostic_reduction_axeswithget_reduction_axesappears to be well-integrated into the codebase, with the latter function being directly tested for its functionality. The absence of the former function in the search results supports the notion that it has been replaced. Based on the information available, it seems reasonable to conclude thatget_reduction_axesis intended to provide compatible or enhanced functionality compared to the previous method, and its usage is supported by tests within the codebase.Scripts Executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash # Verify that get_reduction_axes provides compatible functionality # This script is a placeholder for the actual verification process, which should include: # 1. Reviewing the implementation of get_reduction_axes. # 2. Ensuring it handles all cases previously covered by get_channel_agnostic_reduction_axes. # 3. Testing the new implementation in the context of the ChannelAlignment algorithm. echo "Manual verification needed to ensure get_reduction_axes compatibility."Length of output: 150
Script:
#!/bin/bash # Locate the implementation of get_reduction_axes fd ".*\.py$" --exec rg "def get_reduction_axes" {} # Locate the implementation of get_channel_agnostic_reduction_axes if it still exists fd ".*\.py$" --exec rg "def get_channel_agnostic_reduction_axes" {} # Search for tests related to get_reduction_axes fd ".*test.*\.py$" --exec rg "get_reduction_axes" {}Length of output: 1321
tests/post_training/test_templates/test_channel_alignment.py (2)
- 531-531: The call to
MockBackend.get_statistic_collectorhas been updated with new parameters. Ensure that the new parameters(2, 3), 1e-4, ref_subset_size, ref_inplacealign with the expected functionality and that the changes are consistent with the objectives of enhancing tensor statistics aggregation and reduction functionality. It's important to verify that the new parameters are correctly utilized within theget_statistic_collectormethod and that they contribute to the intended improvements in data handling and processing.- 558-558: The modification of
_aggregation_axesto(0, 1)in thetest_statistic_collectorsmethod changes the axes over which aggregation is performed. This adjustment should be carefully reviewed to ensure it aligns with the intended axis manipulation enhancements described in the PR objectives. It's crucial to verify that this change accurately reflects the new logic for determining reduction axes based on tensor statistics and that it contributes to the overall improvements in the framework's efficiency and flexibility.nncf/quantization/quantize_model.py (2)
- 113-113: The call to
_update_advanced_quantization_parameterswithin thequantizefunction is correctly placed and ensures that advanced quantization parameters are updated based on the batch size of the calibration dataset before proceeding with quantization. This is a good practice for ensuring that the quantization process is tailored to the dataset characteristics.- 248-250: Similar to the
quantizefunction, thequantize_with_accuracy_controlfunction correctly updates the advanced quantization parameters using_update_advanced_quantization_parameters. This consistency across functions is commendable and ensures that the advanced parameters are appropriately handled regardless of the quantization approach used.nncf/torch/graph/operator_metatypes.py (1)
- 1055-1064: The addition of
OPERATIONS_OUTPUT_HAS_NO_BATCH_AXISis a thoughtful approach to addressing potential inaccuracies in statistics collection due to operations that mix outputs for different samples into one axis. It's crucial to keep this list updated as new metatypes are introduced or existing ones are modified to ensure the accuracy of statistics results.tests/common/test_statistics_aggregator.py (7)
- 27-27: The import of
EMPTY_DATASET_MESSAGEis added to support new test cases handling empty datasets. This is a good practice as it leverages a predefined message for consistency across error handling.- 35-35: The import of
MinMaxQuantizationis necessary for the new methodget_min_max_algo_clswhich returns theMinMaxQuantizationclass. This aligns with the PR's objective to refine tensor statistics aggregation and reduction functionality.- 44-50: The
MockedDatasetclass is introduced to simulate a dataset with no elements, particularly useful for testing scenarios where the dataset is empty. This is a straightforward and effective way to test edge cases without needing to manipulate actual datasets.- 132-133: The
get_min_max_algo_clsmethod is added to return theMinMaxQuantizationclass. This method simplifies the process of obtaining the class for quantization algorithms, supporting the PR's goal of improving the quantization process.- 398-398: The addition of the
mockerparameter in thetest_statistics_aggregator_min_maxfunction (and similar additions in other test functions) is crucial for enabling the mocking of dependencies during testing. This enhances the test's reliability by isolating it from external changes and dependencies.- 907-914: The
test_collect_with_empty_datasetmethod correctly tests the behavior of the statistics aggregator when provided with an empty dataset. It ensures that an appropriate error is raised, aligning with the objective of improving dataset validation and batch size warnings.- 916-935: The
test_collect_with_empty_dataset_no_lenmethod tests the scenario where a dataset without alen()method (but effectively empty) is used. This test is important for ensuring robustness in handling datasets that might not conform to typical expectations, such as those lacking alen()method.tests/openvino/tools/calibrate.py (3)
- 104-105: The addition of the
--batch_sizeargument to the command-line parser is correctly implemented with a default value of 1. This change allows users to specify the batch size for data loading, which can improve data handling efficiency.- 1038-1043: The
update_configfunction correctly updates the batch size in the accuracy checker configuration. This function iterates through the models and datasets within the accuracy checker configuration and updates thebatchattribute to the newbatch_sizevalue. This is a straightforward and effective way to propagate the batch size specified by the user to the underlying dataset handling logic.- 1055-1055: The call to
update_configin themainfunction correctly passes theargs.batch_sizeargument to update the batch size in the accuracy checker configuration. This ensures that the user-specified batch size is applied to the datasets used in the calibration process.nncf/quantization/algorithms/min_max/algorithm.py (2)
- 401-444: The
_get_stat_collectormethod has been updated to utilize theis_per_sampleparameter, which is derived from the newly introducedstatistics_per_sample. This change allows for more granular control over how statistics are collected, potentially improving the accuracy of quantization for certain models. However, ensure that the logic correctly handles cases where per-sample statistics collection is not applicable or desired, especially for weight tensors whereis_per_sampleis forcibly set toFalse.Ensure that the implementation correctly handles edge cases and that the performance impact of collecting statistics per sample is acceptable.
- 937-938: The call to
_get_stat_collectorwithin theget_statistic_pointsmethod correctly passes thestatistics_per_sampleparameter. This ensures that the statistic collection behavior is consistent with the configuration specified during the initialization of theMinMaxQuantizationinstance. This change aligns with the objective of enhancing the flexibility of statistics collection based on the new parameter.
Changes
Reason for changes
Related tickets
Tests
Summary by CodeRabbit