Skip to content

Commit

Permalink
Fix reference to GEMMA_WEIGHT_T. Refs #198
Browse files Browse the repository at this point in the history
PiperOrigin-RevId: 641161403
  • Loading branch information
jan-wassenberg authored and copybara-github committed Jun 7, 2024
1 parent 12707ad commit 8dc0e5e
Showing 1 changed file with 7 additions and 6 deletions.
13 changes: 7 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,14 +149,15 @@ For the 8-bit switched floating point weights (sfp), run cmake with no options:
cmake -B build
```

**or** if you downloaded bfloat16 weights (any model *without* `-sfp` in the name),
instead of running cmake with no options as above, run cmake with WEIGHT_TYPE
set to [highway's](https://github.com/google/highway) `hwy::bfloat16_t` type
(this will be simplified in the future, we recommend using `-sfp` weights
instead of bfloat16 for faster inference):
**or** if you downloaded bfloat16 weights (any model *without* `-sfp` in the
name), instead of running cmake with no options as above, run cmake with
GEMMA_WEIGHT_T set to [highway's](https://github.com/google/highway)
`hwy::bfloat16_t` type. We intend to soon support all weight types without
requiring compiler flags. Note that we recommend using `-sfp` weights instead of
bfloat16 for faster inference.

```sh
cmake -B build -DWEIGHT_TYPE=hwy::bfloat16_t
cmake -B build -DGEMMA_WEIGHT_T=hwy::bfloat16_t
```

After running whichever of the above `cmake` invocations that is appropriate for
Expand Down

0 comments on commit 8dc0e5e

Please sign in to comment.