Skip to content

Commit

Permalink
fix: Update README.md with additional details on tuning time calculat…
Browse files Browse the repository at this point in the history
…ion (#528)

**Reason for Change**:
Update README.md with additional details on tuning time calculation

Signed-off-by: Ishaan Sehgal <[email protected]>
Co-authored-by: Fei Guo <[email protected]>
  • Loading branch information
ishaansehgal99 and Fei-Guo authored Jul 19, 2024
1 parent 137154b commit f42a77d
Showing 1 changed file with 12 additions and 1 deletion.
13 changes: 12 additions & 1 deletion docs/tuning/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,17 @@ TrainingArguments [Full List](https://huggingface.co/docs/transformers/v4.40.2/e
- save_strategy: Strategy for saving checkpoints, e.g., "epoch".
- per_device_train_batch_size: Batch size per device during training.
- num_train_epochs: Total number of training epochs to perform, defaults to 3.0.
- Additional Details:
- Number of Steps per Epoch: The number of steps per epoch is determined by the size of the dataset and the batch size:
```
Number of Steps per Epoch = Number of Samples in Dataset / Batch Size
```
You can specify the batch size here with `per_device_train_batch_size`.
- Total Tuning Time: The total tuning time depends on the number of epochs, the batch size, and the max steps. The total number of steps can be calculated as:
```
Total Steps = Number of Epochs * (Number of Samples in Dataset / Batch Size)
```
If `max_steps` is specified, training will stop after reaching this number of steps, even if the specified epochs have not been completed.

DataCollator [Full List](https://huggingface.co/docs/transformers/v4.40.2/en/main_classes/data_collator#transformers.DataCollatorForLanguageModeling)
- mlm: Masked language modeling flag.
Expand Down Expand Up @@ -129,4 +140,4 @@ The SFTTrainer supports popular dataset formats, allowing direct passage of the

To ensure proper function, you may need to preprocess the dataset to match one of the supported formats.

For example usage and more details, refer to the [Official Hugging Face documentation and tutorials](https://huggingface.co/docs/trl/v0.9.4/sft_trainer#dataset-format-support).
For example usage and more details, refer to the [Official Hugging Face documentation and tutorials](https://huggingface.co/docs/trl/v0.9.4/sft_trainer#dataset-format-support).

0 comments on commit f42a77d

Please sign in to comment.