Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use fp32 inference precision #3018

Merged
merged 6 commits into from
Oct 25, 2024
Merged

Use fp32 inference precision #3018

merged 6 commits into from
Oct 25, 2024

Conversation

andrey-churkin
Copy link
Contributor

@andrey-churkin andrey-churkin commented Oct 16, 2024 •
edited
Loading

Changes

  • Use fp32 inference precision

Reason for changes

Ref: 140438

Related tickets

Ref: 140438

@andrey-churkin andrey-churkin requested a review from a team as a code owner October 16, 2024 11:22
@github-actions github-actions bot added NNCF Common Pull request that updates NNCF Common NNCF OpenVINO Pull requests that updates NNCF OpenVINO labels Oct 16, 2024
@andrey-churkin
Copy link
Contributor Author

@alexsu52 @KodiaqQ Are you guys okay with how this was implemented, or are any changes required? Please review, as I will start validation soon.

@andrey-churkin andrey-churkin marked this pull request as draft October 17, 2024 08:24
@github-actions github-actions bot added the NNCF PTQ Pull requests that updates NNCF PTQ label Oct 17, 2024
alexsu52
alexsu52 reviewed Oct 18, 2024
View reviewed changes
nncf/openvino/engine.py Outdated Show resolved Hide resolved
@github-actions github-actions bot removed the NNCF Common Pull request that updates NNCF Common label Oct 18, 2024
@andrey-churkin
Copy link
Contributor Author

Should I use the fp32 hint here as well?

nncf/nncf/quantization/algorithms/weight_compression/openvino_backend.py

Line 348 in dfc5d78

compiled_model = ov.compile_model(model, device_name="CPU")

nncf/nncf/quantization/algorithms/weight_compression/openvino_backend.py

Line 381 in dfc5d78

compiled_model = ov.compile_model(model, device_name="CPU")

@andrey-churkin andrey-churkin marked this pull request as ready for review October 18, 2024 10:30
@andrey-churkin andrey-churkin changed the title Use fp32 precision for statistics collection Use fp32 inference precision Oct 18, 2024
@andrey-churkin andrey-churkin mentioned this pull request Oct 18, 2024
Closed
@andrey-churkin
Copy link
Contributor Author

@alexsu52 @KodiaqQ Please review

@alexsu52
Copy link
Contributor

Should I use the fp32 hint here as well?

nncf/nncf/quantization/algorithms/weight_compression/openvino_backend.py

Line 348 in dfc5d78

compiled_model = ov.compile_model(model, device_name="CPU")

nncf/nncf/quantization/algorithms/weight_compression/openvino_backend.py

Line 381 in dfc5d78

compiled_model = ov.compile_model(model, device_name="CPU")

Yes, you should use the fp32 hint there as well.

alexsu52
alexsu52 previously approved these changes Oct 21, 2024
View reviewed changes
Copy link
Contributor

@alexsu52 alexsu52 left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Please provide PTQ conformance test results.

@alexsu52 alexsu52 dismissed their stale review October 21, 2024 10:36

Please provide PTQ conformance test results.

@andrey-churkin
Copy link
Contributor Author

NNCF/manual/post_training_quantization: (develop) Build # 517
NNCF/manual/post_training_quantization: Build # 516

@alexsu52 alexsu52 merged commit 6afb13d into openvinotoolkit:develop Oct 25, 2024
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@KodiaqQ KodiaqQ KodiaqQ approved these changes

@alexsu52 alexsu52 alexsu52 left review comments

Assignees
No one assigned
Labels
Code Freeze NNCF OpenVINO Pull requests that updates NNCF OpenVINO NNCF PTQ Pull requests that updates NNCF PTQ
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@andrey-churkin @alexsu52 @KodiaqQ