Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AttributeError: 'str' object has no attribute 'requires_grad_' #38

Closed
zhihui-shao opened this issue Jan 10, 2024 · 7 comments
Closed

AttributeError: 'str' object has no attribute 'requires_grad_' #38

zhihui-shao opened this issue Jan 10, 2024 · 7 comments

Comments

@zhihui-shao
Copy link

zhihui-shao commented Jan 10, 2024 •
edited
Loading 

Traceback (most recent call last):
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_mem.py", line 15, in <module>
    train()
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_graph.py", line 863, in train
    model_graph_dict = model.get_model().initialize_graph_modules(
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/model/GraphLlama.py", line 148, in initialize_graph_modules
    graph_tower.requires_grad_(False)
AttributeError: 'str' object has no attribute 'requires_grad_'

我认为可能是vicuna的config.json中的问题：

"graph_hidden_size": 128, 
"pretrain_graph_model_path": "/home/shaozhihui/szh/GraphGPT/graph_transformer/"

我的目录结构如下：
image

这是我的运行命令

model_path=./vicuna-7b-v1.5-16k
instruct_ds=./data/stage_1/graph_matching.json
graph_data_path=./graph_data/all_graph_data.pt
pretra_gnn=./graph_transformer/clip_gt_arxiv_pub.pkl
output_model=./checkpoints/stage_1

wandb offline
CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.run --nnodes=1 --nproc_per_node=2 --master_port=20001 \
    graphgpt/train/train_mem.py \
    --model_name_or_path ${model_path} \
    --version v1 \
    --data_path ${instruct_ds} \
    --graph_content ./arxiv_ti_ab.json \
    --graph_data_path ${graph_data_path} \
    --graph_tower ${pretra_gnn} \
    --tune_graph_mlp_adapter True \
    --graph_select_layer -2 \
    --use_graph_start_end \
    --bf16 False \
    --output_dir ${output_model} \
    --num_train_epochs 3 \
    --per_device_train_batch_size 2 \
    --per_device_eval_batch_size 2 \
    --gradient_accumulation_steps 1 \
    --evaluation_strategy "no" \
    --save_strategy "steps" \
    --save_steps 2400 \
    --save_total_limit 1 \
    --learning_rate 2e-3 \
    --weight_decay 0. \
    --warmup_ratio 0.03 \
    --lr_scheduler_type "cosine" \
    --logging_steps 1 \
    --tf32 False \
    --model_max_length 2048 \
    --gradient_checkpointing True \
    --lazy_preprocess True \
    --report_to wandb

有人能帮帮我吗，谢谢
@linwhitehat
Copy link 

Traceback (most recent call last):
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_mem.py", line 15, in <module>
    train()
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_graph.py", line 863, in train
    model_graph_dict = model.get_model().initialize_graph_modules(
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/model/GraphLlama.py", line 148, in initialize_graph_modules
    graph_tower.requires_grad_(False)
AttributeError: 'str' object has no attribute 'requires_grad_'

我认为可能是vicuna的config.json中的问题：

"graph_hidden_size": 128, 
"pretrain_graph_model_path": "/home/shaozhihui/szh/GraphGPT/graph_transformer/"

我的目录结构如下： image

这是我的运行命令

model_path=./vicuna-7b-v1.5-16k
instruct_ds=./data/stage_1/graph_matching.json
graph_data_path=./graph_data/all_graph_data.pt
pretra_gnn=./graph_transformer/clip_gt_arxiv_pub.pkl
output_model=./checkpoints/stage_1

wandb offline
CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.run --nnodes=1 --nproc_per_node=2 --master_port=20001 \
    graphgpt/train/train_mem.py \
    --model_name_or_path ${model_path} \
    --version v1 \
    --data_path ${instruct_ds} \
    --graph_content ./arxiv_ti_ab.json \
    --graph_data_path ${graph_data_path} \
    --graph_tower ${pretra_gnn} \
    --tune_graph_mlp_adapter True \
    --graph_select_layer -2 \
    --use_graph_start_end \
    --bf16 False \
    --output_dir ${output_model} \
    --num_train_epochs 3 \
    --per_device_train_batch_size 2 \
    --per_device_eval_batch_size 2 \
    --gradient_accumulation_steps 1 \
    --evaluation_strategy "no" \
    --save_strategy "steps" \
    --save_steps 2400 \
    --save_total_limit 1 \
    --learning_rate 2e-3 \
    --weight_decay 0. \
    --warmup_ratio 0.03 \
    --lr_scheduler_type "cosine" \
    --logging_steps 1 \
    --tf32 False \
    --model_max_length 2048 \
    --gradient_checkpointing True \
    --lazy_preprocess True \
    --report_to wandb

有人能帮帮我吗，谢谢

我解决了，作者写的的配置不对，应该改成pretra_gnn=clip_gt_arxiv

@zhihui-shao
Copy link
Author

zhihui-shao commented Jan 13, 2024 •
edited
Loading 

Traceback (most recent call last):
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_mem.py", line 15, in <module>
    train()
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_graph.py", line 863, in train
    model_graph_dict = model.get_model().initialize_graph_modules(
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/model/GraphLlama.py", line 148, in initialize_graph_modules
    graph_tower.requires_grad_(False)
AttributeError: 'str' object has no attribute 'requires_grad_'

我认为可能是vicuna的config.json中的问题：

"graph_hidden_size": 128, 
"pretrain_graph_model_path": "/home/shaozhihui/szh/GraphGPT/graph_transformer/"

我的目录结构如下： image
这是我的运行命令

model_path=./vicuna-7b-v1.5-16k
instruct_ds=./data/stage_1/graph_matching.json
graph_data_path=./graph_data/all_graph_data.pt
pretra_gnn=./graph_transformer/clip_gt_arxiv_pub.pkl
output_model=./checkpoints/stage_1

wandb offline
CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.run --nnodes=1 --nproc_per_node=2 --master_port=20001 \
    graphgpt/train/train_mem.py \
    --model_name_or_path ${model_path} \
    --version v1 \
    --data_path ${instruct_ds} \
    --graph_content ./arxiv_ti_ab.json \
    --graph_data_path ${graph_data_path} \
    --graph_tower ${pretra_gnn} \
    --tune_graph_mlp_adapter True \
    --graph_select_layer -2 \
    --use_graph_start_end \
    --bf16 False \
    --output_dir ${output_model} \
    --num_train_epochs 3 \
    --per_device_train_batch_size 2 \
    --per_device_eval_batch_size 2 \
    --gradient_accumulation_steps 1 \
    --evaluation_strategy "no" \
    --save_strategy "steps" \
    --save_steps 2400 \
    --save_total_limit 1 \
    --learning_rate 2e-3 \
    --weight_decay 0. \
    --warmup_ratio 0.03 \
    --lr_scheduler_type "cosine" \
    --logging_steps 1 \
    --tf32 False \
    --model_max_length 2048 \
    --gradient_checkpointing True \
    --lazy_preprocess True \
    --report_to wandb

有人能帮帮我吗，谢谢

我解决了，作者写的的配置不对，应该改成pretra_gnn=clip_gt_arxiv

我现在修改为pretra_gnn=clip_gt_arxiv_pub，但还是报错了：

Traceback (most recent call last):
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_mem.py", line 13, in <module>
    train()
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_graph.py", line 866, in train
    model_graph_dict = model.get_model().initialize_graph_modules(
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/model/GraphLlama.py", line 145, in initialize_graph_modules
    clip_graph, args= load_model_pretrained(CLIP, self.config.pretrain_graph_model_path) 
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/model/GraphLlama.py", line 54, in load_model_pretrained
    assert osp.exists(osp.join(pretrain_model_path, 'config.json')), 'config.json missing'
AssertionError: config.json missing

我发现我的pretrain_model_path: ./graph_transformer/clip_gt_arxiv_pub，而我的GNN模型目录结构为：
image

我发现train_graph.py的804行model.config.pretrain_graph_model_path = model.config.pretrain_graph_model_path + model_args.graph_tower
它似乎将pretrain_graph_model_path和pretra_gnn拼接在了一起，导致找不到config文件。
请问您知道应该如何解决吗，不胜感激！

@linwhitehat
Copy link 

Traceback (most recent call last):
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_mem.py", line 15, in <module>
    train()
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_graph.py", line 863, in train
    model_graph_dict = model.get_model().initialize_graph_modules(
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/model/GraphLlama.py", line 148, in initialize_graph_modules
    graph_tower.requires_grad_(False)
AttributeError: 'str' object has no attribute 'requires_grad_'

我认为可能是vicuna的config.json中的问题：

"graph_hidden_size": 128, 
"pretrain_graph_model_path": "/home/shaozhihui/szh/GraphGPT/graph_transformer/"

我的目录结构如下： image
这是我的运行命令

model_path=./vicuna-7b-v1.5-16k
instruct_ds=./data/stage_1/graph_matching.json
graph_data_path=./graph_data/all_graph_data.pt
pretra_gnn=./graph_transformer/clip_gt_arxiv_pub.pkl
output_model=./checkpoints/stage_1

wandb offline
CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.run --nnodes=1 --nproc_per_node=2 --master_port=20001 \
    graphgpt/train/train_mem.py \
    --model_name_or_path ${model_path} \
    --version v1 \
    --data_path ${instruct_ds} \
    --graph_content ./arxiv_ti_ab.json \
    --graph_data_path ${graph_data_path} \
    --graph_tower ${pretra_gnn} \
    --tune_graph_mlp_adapter True \
    --graph_select_layer -2 \
    --use_graph_start_end \
    --bf16 False \
    --output_dir ${output_model} \
    --num_train_epochs 3 \
    --per_device_train_batch_size 2 \
    --per_device_eval_batch_size 2 \
    --gradient_accumulation_steps 1 \
    --evaluation_strategy "no" \
    --save_strategy "steps" \
    --save_steps 2400 \
    --save_total_limit 1 \
    --learning_rate 2e-3 \
    --weight_decay 0. \
    --warmup_ratio 0.03 \
    --lr_scheduler_type "cosine" \
    --logging_steps 1 \
    --tf32 False \
    --model_max_length 2048 \
    --gradient_checkpointing True \
    --lazy_preprocess True \
    --report_to wandb

有人能帮帮我吗，谢谢

我解决了，作者写的的配置不对，应该改成pretra_gnn=clip_gt_arxiv

我现在修改为pretra_gnn=clip_gt_arxiv_pub，但还是报错了：

Traceback (most recent call last):
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_mem.py", line 13, in <module>
    train()
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_graph.py", line 866, in train
    model_graph_dict = model.get_model().initialize_graph_modules(
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/model/GraphLlama.py", line 145, in initialize_graph_modules
    clip_graph, args= load_model_pretrained(CLIP, self.config.pretrain_graph_model_path) 
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/model/GraphLlama.py", line 54, in load_model_pretrained
    assert osp.exists(osp.join(pretrain_model_path, 'config.json')), 'config.json missing'
AssertionError: config.json missing

我发现我的pretrain_model_path: ./graph_transformer/clip_gt_arxiv_pub，而我的GNN模型目录结构为： image

我发现train_graph.py的804行model.config.pretrain_graph_model_path = model.config.pretrain_graph_model_path + model_args.graph_tower 它似乎将pretrain_graph_model_path和pretra_gnn拼接在了一起，导致找不到config文件。 请问您知道应该如何解决吗，不胜感激！

Traceback (most recent call last):
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_mem.py", line 15, in <module>
    train()
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_graph.py", line 863, in train
    model_graph_dict = model.get_model().initialize_graph_modules(
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/model/GraphLlama.py", line 148, in initialize_graph_modules
    graph_tower.requires_grad_(False)
AttributeError: 'str' object has no attribute 'requires_grad_'

我认为可能是vicuna的config.json中的问题：

"graph_hidden_size": 128, 
"pretrain_graph_model_path": "/home/shaozhihui/szh/GraphGPT/graph_transformer/"

我的目录结构如下： image
这是我的运行命令

model_path=./vicuna-7b-v1.5-16k
instruct_ds=./data/stage_1/graph_matching.json
graph_data_path=./graph_data/all_graph_data.pt
pretra_gnn=./graph_transformer/clip_gt_arxiv_pub.pkl
output_model=./checkpoints/stage_1

wandb offline
CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.run --nnodes=1 --nproc_per_node=2 --master_port=20001 \
    graphgpt/train/train_mem.py \
    --model_name_or_path ${model_path} \
    --version v1 \
    --data_path ${instruct_ds} \
    --graph_content ./arxiv_ti_ab.json \
    --graph_data_path ${graph_data_path} \
    --graph_tower ${pretra_gnn} \
    --tune_graph_mlp_adapter True \
    --graph_select_layer -2 \
    --use_graph_start_end \
    --bf16 False \
    --output_dir ${output_model} \
    --num_train_epochs 3 \
    --per_device_train_batch_size 2 \
    --per_device_eval_batch_size 2 \
    --gradient_accumulation_steps 1 \
    --evaluation_strategy "no" \
    --save_strategy "steps" \
    --save_steps 2400 \
    --save_total_limit 1 \
    --learning_rate 2e-3 \
    --weight_decay 0. \
    --warmup_ratio 0.03 \
    --lr_scheduler_type "cosine" \
    --logging_steps 1 \
    --tf32 False \
    --model_max_length 2048 \
    --gradient_checkpointing True \
    --lazy_preprocess True \
    --report_to wandb

有人能帮帮我吗，谢谢

我解决了，作者写的的配置不对，应该改成pretra_gnn=clip_gt_arxiv

我现在修改为pretra_gnn=clip_gt_arxiv_pub，但还是报错了：

Traceback (most recent call last):
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_mem.py", line 13, in <module>
    train()
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/train/train_graph.py", line 866, in train
    model_graph_dict = model.get_model().initialize_graph_modules(
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/model/GraphLlama.py", line 145, in initialize_graph_modules
    clip_graph, args= load_model_pretrained(CLIP, self.config.pretrain_graph_model_path) 
  File "/home/shaozhihui/szh/GraphGPT/graphgpt/model/GraphLlama.py", line 54, in load_model_pretrained
    assert osp.exists(osp.join(pretrain_model_path, 'config.json')), 'config.json missing'
AssertionError: config.json missing

我发现我的pretrain_model_path: ./graph_transformer/clip_gt_arxiv_pub，而我的GNN模型目录结构为： image

我发现train_graph.py的804行model.config.pretrain_graph_model_path = model.config.pretrain_graph_model_path + model_args.graph_tower 它似乎将pretrain_graph_model_path和pretra_gnn拼接在了一起，导致找不到config文件。 请问您知道应该如何解决吗，不胜感激！

我查阅了代码，这里不建议使用文件名，而是直接写为“pretra_gnn=clip_gt_arxiv”，它是一个目录的需求同时又固定了判断选项，同时我记得后续读取模型时会将这里的路径作为模型的目录，所以建议在设置graph_transformer作为模型目录时，新建一个命名为clip_gt_arxiv的子目录，将模型和配置拷贝一份进去。这个问题来自于作者设置的环境目录以及判断配置的代码导致的，必须是clip_gt_arxiv以及其他几个固定选项，否则无法读目录，同时目录设置下需要与这个选项关联作为最终模型目录。（细节可自行查阅代码）

@tjb-tech
Copy link
Collaborator

作为

是的，如果需要加入自己的pre train模型，可以通过修改代码来实现。我们后续会加强代码通用逻辑。感谢您的解答！

@Melo-1017
Copy link

Melo-1017 commented Apr 2, 2024 •
edited
Loading

@tjb-tech @linwhitehat 两位好，我也遇到这个问题，我想了解一下vicuna的json中添加的pretrain_graph_model_path和graphgpt_stage1中的pretra_gnn是不是指同一个路径？也就是clip_gt_arxiv的路径？按照这个issue提到的做法，我在GraphGPT下直接新建了clip_gt_arxiv文件夹存放模型。
image
之后运行graphgpt_stage1，但还是提示 AttributeError: 'str' object has no attribute 'requires_grad_'，此时我的命令是：

model_path=/root/nas/models_hf/vicuna-7b-v1.5
instruct_ds=/root/nas/GraphGPT/train_instruct_graphmatch.json
graph_data_path=/root/nas/GraphGPT/graphgpt/graph_data/graph_data_all.pt
pretra_gnn=/root/nas/GraphGPT/clip_gt_arxiv
output_model=/root/nas/GraphGPT/checkpoints/stage_1

vicuna中则是："pretrain_graph_model_path": "/root/nas/GraphGPT/clip_gt_arxiv"
我若修改为pretra_gnn=clip_gt_arxiv，则报错config.json missing
请问我需要怎样配置路径来解决这个问题？

@tigerbunny2023
Copy link

@tjb-tech @linwhitehat 两位好，我也遇到这个问题，我想了解一下vicuna的json中添加的pretrain_graph_model_path和graphgpt_stage1中的pretra_gnn是不是指同一个路径？也就是clip_gt_arxiv的路径？按照这个issue提到的做法，我在GraphGPT下直接新建了clip_gt_arxiv文件夹存放模型。 image 之后运行graphgpt_stage1，但还是提示 AttributeError: 'str' object has no attribute 'requires_grad_'，此时我的命令是：

model_path=/root/nas/models_hf/vicuna-7b-v1.5
instruct_ds=/root/nas/GraphGPT/train_instruct_graphmatch.json
graph_data_path=/root/nas/GraphGPT/graphgpt/graph_data/graph_data_all.pt
pretra_gnn=/root/nas/GraphGPT/clip_gt_arxiv
output_model=/root/nas/GraphGPT/checkpoints/stage_1

vicuna中则是："pretrain_graph_model_path": "/root/nas/GraphGPT/clip_gt_arxiv" 我若修改为pretra_gnn=clip_gt_arxiv，则报错config.json missing 请问我需要怎样配置路径来解决这个问题？

我遇到了一样的问题，请问你解决了吗？

@Melo-1017
Copy link

@tjb-tech @linwhitehat 两位好，我也遇到这个问题，我想了解一下vicuna的json中添加的pretrain_graph_model_path和graphgpt_stage1中的pretra_gnn是不是指同一个路径？也就是clip_gt_arxiv的路径？按照这个issue提到的做法，我在GraphGPT下直接新建了clip_gt_arxiv文件夹存放模型。 image 之后运行graphgpt_stage1，但还是提示 AttributeError: 'str' object has no attribute 'requires_grad_'，此时我的命令是：

model_path=/root/nas/models_hf/vicuna-7b-v1.5
instruct_ds=/root/nas/GraphGPT/train_instruct_graphmatch.json
graph_data_path=/root/nas/GraphGPT/graphgpt/graph_data/graph_data_all.pt
pretra_gnn=/root/nas/GraphGPT/clip_gt_arxiv
output_model=/root/nas/GraphGPT/checkpoints/stage_1

vicuna中则是："pretrain_graph_model_path": "/root/nas/GraphGPT/clip_gt_arxiv" 我若修改为pretra_gnn=clip_gt_arxiv，则报错config.json missing 请问我需要怎样配置路径来解决这个问题？

我遇到了一样的问题，请问你解决了吗？

没有，我已经提了一个新的issue #61

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@linwhitehat @tjb-tech @Melo-1017 @zhihui-shao @tigerbunny2023