Continue the pre-training step

[miguelwon]

Hi,
I don't want to train instructions but just continue the pre-training step. Yes there any example I can quickly use?
Also, I want to use my local dataset. Any info about how should I write the json file?
Thanks

[ZQ-Dev8]

In your config.yml, what you'll do is put "completion" as the type: in the datasets: section, meaning you're intending to train on a dataset containing a column called "text" where the raw text completion data is. If your column is named something else, you can use the field: parameter in the dataset: to override the default.

For the local dataset, just ensure it's a .jsonl file with a column entitled "text", unless you override it as described above. All other columns will be ignored.

[miguelwon]

Thanks!

[mtisz]

Do I only need to have one field in the jsonl file? Only the "text" field is required? And this "text" field should be text with sequence length, say 8K?