Skip to content

Commit

Permalink
Changelog: add explanation r. QLoRA mem savings
Browse files Browse the repository at this point in the history
  • Loading branch information
Titus-von-Koeller committed Jul 24, 2024
1 parent a7c08af commit 9b72679
Showing 1 changed file with 8 additions and 0 deletions.
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
### 0.43.2

This release is quite significant as the QLoRA bug fix big implications for higher `seqlen` and batch sizes.

For each sequence (i.e. batch size increase of one) we expect memory savings of:
- 405B: 39GB for seqlen 1024, and 4888GB for 128k
- 70B: 20.1GB for 1024 and 2516GB for 128k

This was due to activations being unnecessary for frozen parameters, yet the memory for them was still erroneously allocated due to the now fixed bug.

#### Improvements:

- docs: FSDP+QLoRA and CPU install guide (#1211 #1227, thanks @stevhliu)
Expand Down

0 comments on commit 9b72679

Please sign in to comment.