Added rich-based progress bar

[nikita-savelyevv]

Changes

Added nncf.common.logging.track_progress.track() method to replace tqdm for quantization algorithms bars.
This method is an almost exact copy of rich.progress.track() method, but with an addition of an iteration counter. This is kind of a hack, but rich does not provide a way to add custom Column objects to the track() method.

By default rich.progress.track() renders progress bar as

With an addition of customizations this becomes

With this change the quantization pipeline looks like

For iterables without length the progress bar displays as

How it looks in a notebook:

Reason for changes

• User experience improvement
• Avoiding multi-line logs in CI produced by tqdm, for example:
  Statistics collection: 0%| | 0/300 [00:00<?, ?it/s]
  Statistics collection: 0%| | 1/300 [00:00<01:13, 4.09it/s]
  Statistics collection: 1%| | 2/300 [00:00<00:50, 5.87it/s]
  Statistics collection: 1%|▏ | 4/300 [00:00<00:33, 8.79it/s]
  Statistics collection: 2%|▏ | 6/300 [00:00<00:26, 11.30it/s]
  Statistics collection: 3%|▎ | 8/300 [00:00<00:24, 11.88it/s]

[nikita-savelyevv]

run pylint pre-commit tests
run pytorch pre-commit tests
run tensorflow pre-commit tests
run openvino pre-commit tests
run onnx pre-commit tests

[nikita-savelyevv]

Previously @vshampor expressed some concerns on rich compatibility with our own loggers. I've conducted some checks by running pipelines where our loggers (common logger, torch example logger, tf example logger) operate in parallel with the new progress bar from this PR and haven't noticed any conflicts. The existing loggers perform the same.

[AlexanderDokuchaev]

And need add option to use progressbar in async mode to manually update iterations

nncf/tests/post_training/pipelines/image_classification_timm.py

Lines 140 to 156 in 19d7260

	with tqdm.tqdm(total=dataset_size, desc="Validation", disable=disable_tqdm) as pbar:
	def process_result(request, userdata):
	output_data = request.get_output_tensor().data
	predicted_label = np.argmax(output_data, axis=1)
	predictions[userdata] = [predicted_label]
	pbar.update()
	infer_queue.set_callback(process_result)
	for i, (images, target) in enumerate(val_loader):
	# W/A for memory leaks when using torch DataLoader and OpenVINO
	image_copies = copy.deepcopy(images.numpy())
	infer_queue.start_async(image_copies, userdata=i)
	references[i] = target
	infer_queue.wait_all()

[nikita-savelyevv]

Added this functionality by turning track into a class with __iter__, __enter__ and __exit__ implemented similarly to tqdm. It can be used in the following way:

from nncf.common.logging.track_progress import track
n = 10000
with track(total=n, description="Description") as pbar:
    for _ in range(n):
        np.random.random((100000,))
        pbar.progress.update(pbar.task, advance=1)

[codecov]

Codecov Report

Merging #2132 (cf24959) into develop (0a08966) will increase coverage by 0.30%.
Report is 4 commits behind head on develop.
The diff coverage is 88.52%.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #2132      +/-   ##
===========================================
+ Coverage    35.92%   36.23%   +0.30%     
===========================================
  Files          476      477       +1     
  Lines        42483    42544      +61     
===========================================
+ Hits         15264    15414     +150     
+ Misses       27219    27130      -89     

Files Changed	Coverage Δ
setup.py	0.00% <ø> (ø)
.../quantization/algorithms/smooth_quant/algorithm.py	27.84% <50.00%> (ø)
...tization/algorithms/channel_alignment/algorithm.py	25.41% <66.66%> (ø)
nncf/common/logging/track_progress.py	90.00% <90.00%> (ø)
nncf/common/tensor_statistics/aggregator.py	98.14% <100.00%> (ø)
...antization/algorithms/bias_correction/algorithm.py	93.88% <100.00%> (ø)
...ation/algorithms/fast_bias_correction/algorithm.py	90.83% <100.00%> (ø)

... and 6 files with indirect coverage changes

[nikita-savelyevv]

run pylint pre-commit tests
run pytorch pre-commit tests
run tensorflow pre-commit tests
run openvino pre-commit tests
run onnx pre-commit tests