You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Here I used the dataset given by git as a test for the initial fine-tuning, but there was no response, and then I thought that the dataset might be too large, resulting in too long a time to split the validation set and the training set, I did some datasets according to his format for testing, or stuck in Verifying settings... No response for a long time, here is the log from the command line. I wonder if something went wrong with the settings. Thank you very much for your answers!!
Please describe yourAIGC\litgpt\litgpt>litgpt finetune microsoft/phi-2 --data JSON --data.json_path my_custom_dataset.json --data.val_split_fraction 0.1 --out_dir out/custom-model
{'access_token': None,
'checkpoint_dir': WindowsPath('checkpoints/microsoft/phi-2'),
'data': JSON(json_path=WindowsPath('my_custom_dataset.json'),
mask_prompt=False,
val_split_fraction=0.1,
prompt_style=<litgpt.prompts.Alpaca object at 0x0000023E1F336440>,
ignore_index=-100,
seed=42,
num_workers=4),
'devices': 1,
'eval': EvalArgs(interval=100,
max_new_tokens=100,
max_iters=100,
initial_validation=False,
final_validation=True,
evaluate_example='first'),
'logger_name': 'csv',
'lora_alpha': 16,
'lora_dropout': 0.05,
'lora_head': False,
'lora_key': False,
'lora_mlp': False,
'lora_projection': False,
'lora_query': True,
'lora_r': 8,
'lora_value': True,
'num_nodes': 1,
'optimizer': 'AdamW',
'out_dir': WindowsPath('out/custom-model'),
'precision': None,
'quantize': None,
'seed': 1337,
'train': TrainArgs(save_interval=1000,
log_interval=1,
global_batch_size=16,
micro_batch_size=1,
lr_warmup_steps=100,
lr_warmup_fraction=None,
epochs=5,
max_tokens=None,
max_steps=None,
max_seq_length=None,
tie_embeddings=None,
max_norm=None,
min_lr=6e-05)}
Using bfloat16 Automatic Mixed Precision (AMP)
Seed set to 1337
Number of trainable parameters: 2,621,440
Number of non-trainable parameters: 2,779,683,840
The longest sequence length in the train data is 363, the model's maximum sequence length is 363 and context length is 2048
Verifying settings ...
The text was updated successfully, but these errors were encountered:
Here is the device information:
Device name DESKTOP-VAT73EM
Processor Intel(R) Core(TM) i9-14900KF 3.20 GHz
Band RAM 64.0 GB (63.8 GB available)
Graphics card 4090
System type 64-bit operating system, x64-based processor
Disk 2T SSD + 10T HDD
Tuned model microsoft/phi-2
Here I used the dataset given by git as a test for the initial fine-tuning, but there was no response, and then I thought that the dataset might be too large, resulting in too long a time to split the validation set and the training set, I did some datasets according to his format for testing, or stuck in Verifying settings... No response for a long time, here is the log from the command line. I wonder if something went wrong with the settings. Thank you very much for your answers!!
Please describe yourAIGC\litgpt\litgpt>litgpt finetune microsoft/phi-2 --data JSON --data.json_path my_custom_dataset.json --data.val_split_fraction 0.1 --out_dir out/custom-model
{'access_token': None,
'checkpoint_dir': WindowsPath('checkpoints/microsoft/phi-2'),
'data': JSON(json_path=WindowsPath('my_custom_dataset.json'),
mask_prompt=False,
val_split_fraction=0.1,
prompt_style=<litgpt.prompts.Alpaca object at 0x0000023E1F336440>,
ignore_index=-100,
seed=42,
num_workers=4),
'devices': 1,
'eval': EvalArgs(interval=100,
max_new_tokens=100,
max_iters=100,
initial_validation=False,
final_validation=True,
evaluate_example='first'),
'logger_name': 'csv',
'lora_alpha': 16,
'lora_dropout': 0.05,
'lora_head': False,
'lora_key': False,
'lora_mlp': False,
'lora_projection': False,
'lora_query': True,
'lora_r': 8,
'lora_value': True,
'num_nodes': 1,
'optimizer': 'AdamW',
'out_dir': WindowsPath('out/custom-model'),
'precision': None,
'quantize': None,
'seed': 1337,
'train': TrainArgs(save_interval=1000,
log_interval=1,
global_batch_size=16,
micro_batch_size=1,
lr_warmup_steps=100,
lr_warmup_fraction=None,
epochs=5,
max_tokens=None,
max_steps=None,
max_seq_length=None,
tie_embeddings=None,
max_norm=None,
min_lr=6e-05)}
Using bfloat16 Automatic Mixed Precision (AMP)
Seed set to 1337
Number of trainable parameters: 2,621,440
Number of non-trainable parameters: 2,779,683,840
The longest sequence length in the train data is 363, the model's maximum sequence length is 363 and context length is 2048
Verifying settings ...
The text was updated successfully, but these errors were encountered: