Skip to content

Commit

Permalink
Fix uint4 url typo. (#1340)
Browse files Browse the repository at this point in the history
Signed-off-by: Ding, Ke <[email protected]>
  • Loading branch information
kding1 authored Sep 20, 2024
1 parent 1a8ad12 commit a324e7c
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion examples/text-generation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -489,7 +489,7 @@ python run_generation.py \
You can load pre-quantized 4bit models with the argument `--load_quantized_model`.
Currently, uint4 checkpoints and single device are supported.
More information on enabling 4 bit inference in SynapseAI is available here:
https://docs.habana.ai/en/latest/PyTorch/Inference_on_PyTorch/Inference_Using_INT4.html.
https://docs.habana.ai/en/latest/PyTorch/Inference_on_PyTorch/Inference_Using_UINT4.html.

Below is an example to load a model with 4bit checkpoints from Hugging Face.
Please note that model name is denoted as `<model_path_in_hugging_face>`.
Expand Down

0 comments on commit a324e7c

Please sign in to comment.