Skip to content

support for LLM.int8
Compare
Choose a tag to compare
@pszemraj pszemraj released this 31 Jan 04:11
· 2 commits to main since this release
v0.1.5
9108f66
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

On GPU, you can now use LLM.int8 to use less memory:

from textsum.summarize import Summarizer
summarizer = Summarizer(load_in_8bit=True) # loads default model in LLM.int8, taking 1/4 of the memory

What's Changed

Full Changelog: v0.1.3...v0.1.5

Contributors

pszemraj
Assets 2
Loading