Skip to content

Commit

Permalink
GPTQ + Scale Eestimation conformance test (#2942)
Browse files Browse the repository at this point in the history
### Changes

replaced tinyllama_data_aware_gptq test by
tinyllama_data_aware_gptq_scale_estimation_stateful test in the weight
compression conformance test

### Reason for changes

Test regression in the GPTQ + Scale Estimation pipeline

### Related tickets

ref 143637

### Tests

tinyllama_data_aware_gptq_scale_estimation_stateful
NNCF/job/manual/job/post_training_weight_compression/226
  • Loading branch information
alexsu52 authored Oct 28, 2024
1 parent dca2cad commit c01a50f
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 4 deletions.
6 changes: 3 additions & 3 deletions tests/post_training/data/wc_reference_data.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@ tinyllama_int8_data_free_backend_TORCH:
metric_value: 0.95624
num_int4: 0
num_int8: 312
tinyllama_data_aware_gptq_backend_OV:
metric_value: 0.86521
tinyllama_data_aware_gptq_scale_estimation_stateful_backend_OV:
metric_value: 0.86697
num_int4: 94
num_int8: 124
metrics_xfail_reason: "Issue-148819"
Expand All @@ -43,4 +43,4 @@ tinyllama_NF4_scale_estimation_stateful_per_channel_backend_OV:
tinyllama_awq_backup_mode_none_backend_OV:
metric_value: 0.84793
num_int4: 208
num_int8: 0
num_int8: 0
7 changes: 6 additions & 1 deletion tests/post_training/model_scope.py
Original file line number Diff line number Diff line change
Expand Up @@ -452,15 +452,20 @@
"backends": [BackendType.TORCH],
},
{
"reported_name": "tinyllama_data_aware_gptq",
"reported_name": "tinyllama_data_aware_gptq_scale_estimation_stateful",
"model_id": "tinyllama/tinyllama-1.1b-step-50k-105b",
"pipeline_cls": LMWeightCompression,
"compression_params": {
"group_size": 64,
"ratio": 0.8,
"mode": CompressWeightsMode.INT4_SYM,
"gptq": True,
"scale_estimation": True,
"advanced_parameters": AdvancedCompressionParameters(
scale_estimation_params=AdvancedScaleEstimationParameters(32, 5, 10, 1.0)
),
},
"params": {"is_stateful": True},
"backends": [BackendType.OV],
},
{
Expand Down

0 comments on commit c01a50f

Please sign in to comment.