Skip to content

Commit

Permalink
fix bug of backend (#294)
Browse files Browse the repository at this point in the history
  • Loading branch information
wenhuach21 authored Oct 29, 2024
1 parent 487abd6 commit 4f22871
Show file tree
Hide file tree
Showing 2 changed files with 3 additions and 3 deletions.
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,14 +57,14 @@ pip install auto-round
### Basic Usage (Gaudi2/CPU/GPU)

A user guide detailing the full list of supported arguments is provided by calling ```auto-round -h``` on the terminal.
Alternatively, you can use ```auto_round``` instead of ```auto-round```.
Alternatively, you can use ```auto_round``` instead of ```auto-round```. Set the format you want in `format` and multiple formats exporting has been supported.

```bash
CUDA_VISIBLE_DEVICES=0 auto-round \
--model facebook/opt-125m \
--bits 4 \
--group_size 128 \
--format auto_round \
--format "auto_round,auto_gptq" \
--disable_eval \
--output_dir ./tmp_autoround
```
Expand Down
2 changes: 1 addition & 1 deletion auto_round/backend.py
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ def feature_num_greater_checker(in_feature, out_feature, num):
requirements=["intel-extension-for-transformers"])

BackendInfos['auto_round:qbits_zp'] = BackendInfo(device=["cpu"], sym=[True, False],
packing_format="qbits",
packing_format="qbits_zp+-1",
bits=[2, 4, 8], group_size=None,
priority=0 if "intel" in get_cpu_manufacturer() else 5,
feature_checks=[],
Expand Down

0 comments on commit 4f22871

Please sign in to comment.