v2.5.6 includes quick Fixes

Assets 2

04 Oct 22:16

Goekdeniz-Guelmez

v2.5.5

00cf104

v2.5.5

KANama is now a PyPi Package!

you can install it via pip install kanama

Changes and Updates:

The KANamav5 MoE model is now renamed to KANaMoEv1.
You can now save and load a model based on the config.json file via calling if from model.handler import save_pretrained, from_pretrained. The save_pretrained has the path and the model as the input, the from_pretrained function only takes in the math of the model and config.json file.
quick_inference has also been moved to model.handler.

Example:

from model.handler import save_pretrained, from_pretrained, quick_inference

# load the model from a path
model = from_pretrained('path/to/model/')

# get a tokenizer and tokenize your prompt

# save the same model to a path
save_pretrained('path/to/new/model/', model)

Assets 2

02 Sep 19:55

Goekdeniz-Guelmez

v2.5.0

9c2a56b

v2.5.0

Adding a MOE model (KANamav5)
Adding a better trainer and fixing batching function
Adding more examples, with tiny-shakespear.txt and fineweb.jsonl datasets
Adding loading bars to the SFTTrainer
General fixes

Assets 2

31 Jul 22:46

Goekdeniz-Guelmez

v1.9.5

cd74cc7

v1.9.5

v1.9.5 with PyPi support

install it via `pip install KANama==1.9.5`

To use it with python

from KANama.model.args import ModelArgs
from KANama.model.KANamav4 import KANamav4

# vocab_size, pad_id must be set
ModelArgs.vocab_size = 30
ModelArgs.pad_id = 0

model = KANamav4(ModelArgs)
print(model)