Skip to content

Commit

Permalink
README: updates info
Browse files Browse the repository at this point in the history
  • Loading branch information
jonasfroeller committed Oct 29, 2024
1 parent d78869b commit 32fe85e
Showing 1 changed file with 3 additions and 2 deletions.
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
# Neptun AI

NeptunAI on [huggingface.co](https://huggingface.co/neptun-org).
NeptunAI on [huggingface.co](https://huggingface.co/neptun-org).
Based on [RWKV-LM](https://github.com/BlinkDL/RWKV-LM/tree/main?tab=readme-ov-file).

## Fine Tuning

1. Use `.jsonl` format for your data (see [rwkv-5-world](https://huggingface.co/BlinkDL/rwkv-5-world) for formats).

2. Use `./RWKV/FineTuning/make_data.py` to tokenize it into `binidx` using the world tokenizer, suitable for fine-tuning world models.
2. Use `./RWKV/FineTuning/make_data.py` to tokenize it into `bin` and `idx` using the world tokenizer, suitable for fine-tuning world models.

3. Rename the base checkpoint in the model folder to `rwkv-init.pth`, and change the training commands to use:

0 comments on commit 32fe85e

Please sign in to comment.