Tensor for PTQ #2058

AlexanderDokuchaev · 2023-08-18T00:45:47Z

Changes

Update MinMax and FastBiasCorrection to use common Tensor.
Remove converting torch -> numpy -> torch of data.
FakeQuantizeParameters collect data wrapped by tensor.
Add support cuda for torch backend.
Add new functions for Tensor:
- stack
- unstack
- moveaxis
- mean
- round
Removed __all__ from function.py, it's works like default behavior.
Add statistical_functions.py for high level functions that used only function from functions.py and have no backend specific implementations:
- mean_per_channel
Disable warnings for divide operators of numpy

Related tickets

113315

Tests

codecov · 2023-08-18T17:49:24Z

Codecov Report

Merging #2058 (d5de2a4) into develop (eaddf18) will increase coverage by 0.19%.
Report is 9 commits behind head on develop.
The diff coverage is 83.43%.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #2058      +/-   ##
===========================================
+ Coverage    36.13%   36.33%   +0.19%     
===========================================
  Files          480      480              
  Lines        42998    43223     +225     
===========================================
+ Hits         15539    15703     +164     
- Misses       27459    27520      +61

Files	Coverage Δ
.../common/tensor_statistics/statistical_functions.py	`100.00% <100.00%> (ø)`
nncf/experimental/tensor/numpy_functions.py	`100.00% <100.00%> (ø)`
nncf/experimental/tensor/tensor.py	`98.03% <100.00%> (+0.10%)`	⬆️
nncf/onnx/quantization/quantizer_parameters.py	`88.88% <100.00%> (ø)`
...ation/algorithms/fast_bias_correction/algorithm.py	`91.54% <100.00%> (+0.70%)`	⬆️
...ization/algorithms/fast_bias_correction/backend.py	`100.00% <100.00%> (ø)`
...on/algorithms/fast_bias_correction/onnx_backend.py	`97.05% <100.00%> (-0.59%)`	⬇️
...cf/quantization/algorithms/min_max/onnx_backend.py	`99.23% <100.00%> (ø)`
...uantization/algorithms/min_max/openvino_backend.py	`0.00% <ø> (ø)`
nncf/experimental/tensor/functions.py	`99.24% <98.24%> (-0.76%)`	⬇️
... and 6 more

... and 15 files with indirect coverage changes

andrey-churkin · 2023-08-30T07:38:32Z

nncf/experimental/tensor/functions.py

-def squeeze(a: TTensor, axis: Optional[Union[int, Tuple[int]]] = None) -> TTensor:
+def squeeze(a: TTensor, axis: Optional[Union[int, Tuple[int]]] = None) -> Tensor:


Could you please explain why do you change this?

Functions in this file always return Tensor or list of Tensor

I think this is not entirely correct. I mean these functions return a Tensor object only if there is no registered version for the type of the passed argument.

Update all return type to Tensor to correctly works "pop-up suggestions" (or how it names when editor suggest functions by first symbols?).
So set Tensor type to first argument. But not sure about second, added like Union[torch.Tensor, float], because it can be used like torch.tensor(1) + 1 or fns.min(torch.tensor(1), 0). May be do you have anny suugestion about it?

nncf/quantization/algorithms/fast_bias_correction/algorithm.py

andrey-churkin · 2023-08-30T07:51:30Z

nncf/quantization/algorithms/fast_bias_correction/algorithm.py

@@ -223,7 +245,7 @@ def input_filter_func(point):
            node_name, input_filter_func, self._algorithm_key
        ):
            statistics = tensor_collector.get_statistics()
-            input_fp.extend(statistics.mean_values)
+            input_fp.extend(Tensor(statistics.mean_values))


I think the tensor_collector should return already wrapped tensor.

Yes, but i dont want to refactor all in one PR. In this PR fixed ptq for cuda and shows how to use Tensor on real code.

IMO, we should provide the changes in case of UX improvements or developing/code support improvements.
Providing changes only for demonstration is not good.
So, this is why I still can't understand why should we add a new wrapper for all tensors in this PR.

AlexanderDokuchaev · 2023-08-30T11:54:33Z

post_training_quantization/141/

AlexanderDokuchaev · 2023-09-29T12:43:22Z

post_training_quantization/159

KodiaqQ

One of the major changes that are not clear:

Why shouldn't we place all tensor-related entities in the one namespace (Tensor, functions);
Why the one algorithm updated with the Tensor and the others are not;
Why the developers should use multiple imports to start working with the Tensor instead of import numpy as np.

KodiaqQ · 2023-10-02T11:07:07Z

nncf/quantization/algorithms/fast_bias_correction/algorithm.py

+from nncf.experimental.common.tensor_statistics import statistical_functions as s_fns
+from nncf.experimental.tensor import Tensor
+from nncf.experimental.tensor import functions as fns


I still don't get when the fns and s_fns would be merged.

nncf/quantization/algorithms/min_max/torch_backend.py

nncf/experimental/tensor/functions.py

nncf/experimental/tensor/__init__.py

alexsu52 · 2023-10-04T06:15:53Z

nncf/quantization/algorithms/fast_bias_correction/algorithm.py

+from nncf.experimental.common.tensor_statistics import statistical_functions as s_fns
+from nncf.experimental.tensor import Tensor
+from nncf.experimental.tensor import functions as fns


What is reason to merge them?

vshampor

I'd like to see the _dispatch_list improved as directed - after that I don't see major blockers for merging this PR (other than the already visible deficiencies of this approach code-wise as compared to the proper OOP approach)

Changes addressed

alexsu52

Please, address my minor comment

nncf/quantization/algorithms/fast_bias_correction/backend.py

AlexanderDokuchaev added 7 commits August 5, 2023 18:12

Add amax, amin, unstack functions

17e91ca

unstack

d87a634

unused func

20b75f6

moveaxis

d82af3d

mean

427ffa3

round

4eefd0e

tensor for bc

5038dab

AlexanderDokuchaev requested a review from a team as a code owner August 18, 2023 00:45

github-actions bot added NNCF PT Pull requests that updates NNCF PyTorch experimental NNCF OpenVINO Pull requests that updates NNCF OpenVINO NNCF ONNX Pull requests that updates NNCF ONNX NNCF PTQ Pull requests that updates NNCF PTQ labels Aug 18, 2023

vshampor added the API Public API-impacting changes label Aug 18, 2023

AlexanderDokuchaev added 2 commits August 18, 2023 05:33

tensor fbc

113a047

linter

519a2e6

AlexanderDokuchaev added 2 commits August 18, 2023 21:46

fix pt

03b1bc1

linter

82c699b

AlexanderDokuchaev force-pushed the ad/ptq_tensor branch from 7d1429d to 9cc7582 Compare August 18, 2023 21:27

AlexanderDokuchaev added 5 commits August 19, 2023 00:31

Merge branch 'develop' into ad/ptq_tensor

cc5bc75

fix

982f579

fix

cf47cbd

fix

9cc7582

fix

840cb6a

AlexanderDokuchaev requested review from alexsu52, KodiaqQ and andrey-churkin August 19, 2023 11:07

andrey-churkin requested changes Aug 30, 2023

View reviewed changes

AlexanderDokuchaev requested review from KodiaqQ and alexsu52 September 27, 2023 14:03

AlexanderDokuchaev added 7 commits September 27, 2023 17:21

mean_per_channel

cfd3f04

_dispatch_list

11c6cba

test_fn_mean_per_channel_incorrect_axis

63d7a16

_dispatch_list in readme

6d62e44

lint

9767308

Merge branch 'develop' into ad/ptq_tensor

5393fa3

Add check to device in tests

35ee9a4

KodiaqQ reviewed Oct 2, 2023

View reviewed changes

andrey-churkin approved these changes Oct 2, 2023

View reviewed changes

AlexanderDokuchaev force-pushed the ad/ptq_tensor branch from 04cd1dc to 35ee9a4 Compare October 2, 2023 13:53

KodiaqQ approved these changes Oct 2, 2023

View reviewed changes

vshampor reviewed Oct 2, 2023

View reviewed changes

nncf/experimental/tensor/functions.py Outdated Show resolved Hide resolved

AlexanderDokuchaev added 4 commits October 2, 2023 19:44

functions to tensor namespace

19640ad

update import mean_per_channel

3ac8523

fix

94ed9a1

linter

4090048

alexsu52 reviewed Oct 4, 2023

View reviewed changes

-

1feb2f5

vshampor self-requested a review October 4, 2023 14:21

Merge branch 'develop' into ad/ptq_tensor

4988067

vshampor previously requested changes Oct 4, 2023

View reviewed changes

dispath_list

43eeba0

AlexanderDokuchaev assigned alexsu52 Oct 6, 2023

alexsu52 approved these changes Oct 9, 2023

View reviewed changes

nncf/quantization/algorithms/fast_bias_correction/backend.py Show resolved Hide resolved

typehint

d5de2a4

alexsu52 merged commit bbb7e56 into openvinotoolkit:develop Oct 9, 2023
11 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tensor for PTQ #2058

Tensor for PTQ #2058

AlexanderDokuchaev commented Aug 18, 2023 •

edited

Loading

codecov bot commented Aug 18, 2023 •

edited

Loading

andrey-churkin Aug 30, 2023

AlexanderDokuchaev Aug 30, 2023

andrey-churkin Aug 31, 2023

AlexanderDokuchaev Sep 26, 2023

andrey-churkin Aug 30, 2023

AlexanderDokuchaev Aug 30, 2023

KodiaqQ Sep 12, 2023

AlexanderDokuchaev commented Aug 30, 2023

AlexanderDokuchaev commented Sep 29, 2023

KodiaqQ left a comment

KodiaqQ Oct 2, 2023

alexsu52 Oct 4, 2023

vshampor left a comment

alexsu52 left a comment

		def squeeze(a: TTensor, axis: Optional[Union[int, Tuple[int]]] = None) -> TTensor:
		def squeeze(a: TTensor, axis: Optional[Union[int, Tuple[int]]] = None) -> Tensor:

Tensor for PTQ #2058

Tensor for PTQ #2058

Conversation

AlexanderDokuchaev commented Aug 18, 2023 • edited Loading

Changes

Related tickets

Tests

codecov bot commented Aug 18, 2023 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AlexanderDokuchaev commented Aug 30, 2023

AlexanderDokuchaev commented Sep 29, 2023

KodiaqQ left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vshampor left a comment

Choose a reason for hiding this comment

alexsu52 left a comment

Choose a reason for hiding this comment

AlexanderDokuchaev commented Aug 18, 2023 •

edited

Loading

codecov bot commented Aug 18, 2023 •

edited

Loading