[PTQ][OV] ChannelAlignment algorithm is ready to be enabled by default #2073

daniil-lyakhov · 2023-08-21T10:55:28Z

Changes

ChannelAlignment algorithm is enabled by default
Biases are added only for operations that are affected by CA algorithm

Reason for changes

To increase models mertics by using ChannelAlignment algorithm by default

Related tickets

114328
114583

Tests

tests/post_training/test_templates/test_channel_alignment.py is updated

codecov · 2023-08-21T11:19:43Z

Codecov Report

Merging #2073 (79a681d) into develop (f6d958f) will increase coverage by 35.93%.
The diff coverage is 46.66%.

Additional details and impacted files

@@             Coverage Diff              @@
##           develop    #2073       +/-   ##
============================================
+ Coverage         0   35.93%   +35.93%     
============================================
  Files            0      476      +476     
  Lines            0    42446    +42446     
============================================
+ Hits             0    15255    +15255     
- Misses           0    27191    +27191

Files Changed	Coverage Δ
...n/algorithms/channel_alignment/openvino_backend.py	`0.00% <0.00%> (ø)`
...tization/algorithms/channel_alignment/algorithm.py	`27.91% <15.38%> (ø)`
...f/common/graph/transformations/command_creation.py	`100.00% <100.00%> (ø)`
nncf/quantization/advanced_parameters.py	`46.04% <100.00%> (ø)`
...antization/algorithms/channel_alignment/backend.py	`100.00% <100.00%> (ø)`
...quantization/algorithms/post_training/algorithm.py	`82.65% <100.00%> (ø)`

... and 470 files with indirect coverage changes

daniil-lyakhov · 2023-08-23T12:31:41Z

Full validation showed no degradation with CA algorithm enabled for supported models.
Conformance test showed no degradation as well

	Model	Backend	Metric name	Metric value	Metric diff	Num FQ	RAM MiB	Quant. time	Total time
1	timm/efficientnet_b0	OV	Acc@1	0.7682	-0.0047	76	1001	0:00:40	0:13:47
2	timm/efficientnet_b0_BC	OV	Acc@1	0.7682	-0.0047	76	1195	0:00:55	0:14:07
3	timm/efficientnet_lite0	OV	Acc@1	0.7471	-0.0025	60	961	0:00:29	0:11:27
4	timm/mobilenetv2_050_BC	OV	Acc@1	0.6436	-0.0020	64	835	0:00:22	0:09:08
5	timm/mobilenetv3_small_050	OV	Acc@1	0.3879	-0.1755	80	737	0:00:22	0:13:17

timm/mobilenetv3_small_050: Improvement: Metric value is better than reference 0.38792 > 0.38638

daniil-lyakhov · 2023-08-25T12:03:02Z

@KodiaqQ, @andrey-churkin, please take a look

nncf/common/graph/transformations/command_creation.py

nncf/quantization/algorithms/channel_alignment/algorithm.py

nncf/quantization/algorithms/channel_alignment/backend.py

alexsu52 · 2023-09-01T06:25:48Z

nncf/quantization/advanced_parameters.py

@@ -152,7 +152,7 @@ class AdvancedQuantizationParameters:
    overflow_fix: OverflowFix = OverflowFix.FIRST_LAYER
    quantize_outputs: bool = False
    inplace_statistics: bool = True
-    disable_channel_alignment: bool = True
+    disable_channel_alignment: bool = False


Please, discuss influence of this change on nncf.quantize_with_accuracy_control(...) with @andrey-churkin.

These changes do not affect nncf.quantize_with_accuracy_control(...) when tune_hyperparams is set to False. However, when tune_hyperparams is set to True, we cannot apply hyperparameter tuning due to this check.

nncf/nncf/quantization/algorithms/post_training/algorithm.py

Lines 166 to 170 in 2fe6ee3

def get_statistic_points(self, model: TModel, graph: NNCFGraph) -> StatisticPointsContainer:

if self.first_stage_algorithms:

raise NotImplementedError(

"Statistic points are not supported yet for SmoothQuant and ChannelAlignment algorithms."

)

@andrey-churkin, what should I do to make tune_hyperparams works again?

andrey-churkin · 2023-09-05T06:05:57Z

nncf/quantization/algorithms/post_training/algorithm.py

-        for first_stage_algorithm in self.first_stage_algorithms:
-            algorithm = first_stage_algorithm.algorithm
-
+        for algorithm in self.first_stage_algorithms:


Are there dependencies in the order of algorithms? I mean, if I apply ChannelAlignment first and then SmoothQuant, does something change?

I've checked and it looks like the order of this algorithms does not matter

…nt_improvements_full

daniil-lyakhov · 2023-09-06T09:40:31Z

@alexsu52, @andrey-churkin, I've disabled CA by default to prevent Tuning Hyperparameters failing. I suggest to merge this PR as it's including some improvements for CA algorithm

alexsu52 · 2023-09-07T07:47:34Z

@alexsu52, @andrey-churkin, I've disabled CA by default to prevent Tuning Hyperparameters failing. I suggest to merge this PR as it's including some improvements for CA algorithm

I agree. Please update the PR title.

alexsu52

LGTM

daniil-lyakhov requested review from KodiaqQ and andrey-churkin August 21, 2023 10:55

daniil-lyakhov requested a review from a team as a code owner August 21, 2023 10:55

github-actions bot added NNCF Common Pull request that updates NNCF Common NNCF OpenVINO Pull requests that updates NNCF OpenVINO NNCF PTQ Pull requests that updates NNCF PTQ labels Aug 21, 2023

daniil-lyakhov mentioned this pull request Aug 24, 2023

[PTQ][OV] Weights layout in layer attributes is introduced #2082

Merged

daniil-lyakhov force-pushed the dl/channel_alignment_improvements_full branch 3 times, most recently from 1eb303f to dd84e18 Compare August 25, 2023 11:19

daniil-lyakhov added 2 commits August 25, 2023 13:22

Enable ChannelAlignment algorithm by default

dd84e18

Fix openvino import in common algo

0dcca5a

daniil-lyakhov requested a review from alexsu52 August 28, 2023 09:33

KodiaqQ reviewed Aug 30, 2023

View reviewed changes

nncf/common/graph/transformations/command_creation.py Outdated Show resolved Hide resolved

nncf/quantization/algorithms/channel_alignment/algorithm.py Show resolved Hide resolved

nncf/quantization/algorithms/channel_alignment/backend.py Outdated Show resolved Hide resolved

alexsu52 reviewed Sep 1, 2023

View reviewed changes

andrey-churkin reviewed Sep 5, 2023

View reviewed changes

daniil-lyakhov requested review from alexsu52, andrey-churkin and KodiaqQ September 5, 2023 19:30

daniil-lyakhov added 2 commits September 5, 2023 21:33

Comments

2723d04

Merge remote-tracking branch 'origin/develop' into dl/channel_alignme…

79a681d

…nt_improvements_full

daniil-lyakhov changed the title ~~[PTQ][OV] Enable ChannelAlignment algorithm by default~~ [PTQ][OV] ChannelAlignment algorithm is ready to be enabled by default Sep 6, 2023

Channel Alignment disabled by default

c653e83

alexsu52 reviewed Sep 7, 2023

View reviewed changes

alexsu52 approved these changes Sep 7, 2023

View reviewed changes

andrey-churkin approved these changes Sep 7, 2023

View reviewed changes

andrey-churkin assigned KodiaqQ Sep 7, 2023

KodiaqQ approved these changes Sep 7, 2023

View reviewed changes

KodiaqQ merged commit c07810b into openvinotoolkit:develop Sep 7, 2023
6 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[PTQ][OV] ChannelAlignment algorithm is ready to be enabled by default #2073

[PTQ][OV] ChannelAlignment algorithm is ready to be enabled by default #2073

daniil-lyakhov commented Aug 21, 2023 •

edited

Loading

codecov bot commented Aug 21, 2023 •

edited

Loading

daniil-lyakhov commented Aug 23, 2023

daniil-lyakhov commented Aug 25, 2023

alexsu52 Sep 1, 2023

andrey-churkin Sep 5, 2023

daniil-lyakhov Sep 5, 2023

andrey-churkin Sep 5, 2023

daniil-lyakhov Sep 5, 2023

daniil-lyakhov commented Sep 6, 2023

alexsu52 commented Sep 7, 2023 •

edited

Loading

alexsu52 left a comment

	def get_statistic_points(self, model: TModel, graph: NNCFGraph) -> StatisticPointsContainer:
	if self.first_stage_algorithms:
	raise NotImplementedError(
	"Statistic points are not supported yet for SmoothQuant and ChannelAlignment algorithms."
	)

[PTQ][OV] ChannelAlignment algorithm is ready to be enabled by default #2073

[PTQ][OV] ChannelAlignment algorithm is ready to be enabled by default #2073

Conversation

daniil-lyakhov commented Aug 21, 2023 • edited Loading

Changes

Reason for changes

Related tickets

Tests

codecov bot commented Aug 21, 2023 • edited Loading

Codecov Report

daniil-lyakhov commented Aug 23, 2023

daniil-lyakhov commented Aug 25, 2023

alexsu52 Sep 1, 2023

Choose a reason for hiding this comment

andrey-churkin Sep 5, 2023

Choose a reason for hiding this comment

daniil-lyakhov Sep 5, 2023

Choose a reason for hiding this comment

andrey-churkin Sep 5, 2023

Choose a reason for hiding this comment

daniil-lyakhov Sep 5, 2023

Choose a reason for hiding this comment

daniil-lyakhov commented Sep 6, 2023

alexsu52 commented Sep 7, 2023 • edited Loading

alexsu52 left a comment

Choose a reason for hiding this comment

daniil-lyakhov commented Aug 21, 2023 •

edited

Loading

codecov bot commented Aug 21, 2023 •

edited

Loading

alexsu52 commented Sep 7, 2023 •

edited

Loading