Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Divide as a * (1.0 / b) during weight compression #3055

Open
wants to merge 21 commits into
base: develop
Choose a base branch
from

Conversation

nikita-savelyevv
Copy link
Collaborator

@nikita-savelyevv nikita-savelyevv commented Nov 1, 2024

Changes

  • Introduce fns.reciprocal(x) = 1.0 / x operation
  • Replace regular division a / b with a * fns.reciprocal(b) during weight compression -related computations.
Model Develop PPL Branch PPL Develop Similarity Branch Similarity
phi3-mini int4_asym 10.96023 10.95891 (-0.012%) 0.89861 0.89635 (-0.25%)
phi3-mini int4_asym SE 10.59984 10.60182 (+0.019%) 0.88692 0.88924 (+0.25%)
phi3-mini int4_asym GPTQ 10.73838 10.72351 (-0.139%) 0.88555 0.86824 (-1.96%)
phi3-mini int4_asym GPTQ + SE 10.51594 10.50970 (-0.059%) 0.871897 0.89654 (+2.82%)

Reason for changes

Align with how division is performed in OpenVINO. Prerequisite to #2727. This should be merged first in order to later track only the differences introduced by the OV computations.

Related tickets

139047

Tests

Added a test comparing inverted division result to a reference value.
NNCF/job/manual/job/post_training_weight_compression/271/
NNCF/job/manual/job/post_training_quantization/576/

Also weekly validation is requested in 158012

@github-actions github-actions bot added NNCF OpenVINO Pull requests that updates NNCF OpenVINO NNCF PTQ Pull requests that updates NNCF PTQ labels Nov 4, 2024
@nikita-savelyevv nikita-savelyevv changed the title Invert Tensor division for __truediv__ and __rtruediv__ Divide as a * (1.0 / b) during weight compression Nov 15, 2024
@nikita-savelyevv
Copy link
Collaborator Author

nikita-savelyevv commented Nov 18, 2024

NNCF/job/manual/job/post_training_weight_compression/267/ has passed.

@nikita-savelyevv nikita-savelyevv marked this pull request as ready for review November 18, 2024 13:00
@nikita-savelyevv nikita-savelyevv requested a review from a team as a code owner November 18, 2024 13:00
Copy link
Contributor

@alexsu52 alexsu52 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@nikita-savelyevv nikita-savelyevv added the do not merge Should not be merged yet label Nov 21, 2024
@alexsu52 alexsu52 assigned alexsu52 and unassigned alexsu52 Nov 22, 2024
@nikita-savelyevv
Copy link
Collaborator Author

The problem with division is resolved by aligning OV division with NP division by disabling ConvertDivide and ConvertDivideWithConstant transformations. These transformation can be disabled for target division nodes by setting "nonconvertable_divide_0" rt_info flag.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
do not merge Should not be merged yet NNCF OpenVINO Pull requests that updates NNCF OpenVINO NNCF PTQ Pull requests that updates NNCF PTQ
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants