Support Lora or other peft

[2033329616]

Do model parallelism and pipeline parallelism support efficient fine-tuning methods such as Lora

[aoyulong]

Flagscale only support the fine-tuning for all parameters for now, but we plan to incorporate the LoRA. Could you provide more detailed requirements so that we can take them into consideration for the future implementation?

[2033329616]

PEFT includes various fine-tuning methods, such as Lora, which is better if the project is compatible with PEFT.

[github-actions]

Marking as stale. No activity in 60 days.