results of motif+DPL

[Zhuzi24]

Hello, after running the experiment I found that the accuracy is very low, I have put the experiment parameters as well as the results below, can you see what the problem is?
train.sh is

export gpu_num=1
export EXP=checkpoints
OUTPATH=$EXP/VG/motif/predcls/DPL
mkdir -p $OUTPATH
export CUDA_VISIBLE_DEVICES=2

python3 tools/relation_train_net.py --config-file /media/dell/DATA/WTZ/DPL-master/configs/e2e_relation_X_101_32_8_FPN_1x.yaml
MODEL.ROI_RELATION_HEAD.USE_GT_BOX True
MODEL.ROI_RELATION_HEAD.USE_GT_OBJECT_LABEL True
OUTPUT_DIR $OUTPATH
MODEL.ROI_RELATION_HEAD.PREDICTOR MotifsLikePredictor_DPL
GLOBAL_SETTING.BASIC_ENCODER Motifs
SOLVER.IMS_PER_BATCH $(expr 3 * $gpu_num)
TEST.IMS_PER_BATCH $(expr 3 * $gpu_num)
SOLVER.MAX_ITER 60000
SOLVER.VAL_PERIOD 2000
SOLVER.CHECKPOINT_PERIOD 2000
MODEL.ROI_RELATION_HEAD.DPL.N_DIM 128
MODEL.ROI_RELATION_HEAD.DPL.ALPHA 10
MODEL.ROI_RELATION_HEAD.DPL.AVG_NUM_SAMPLE 20
MODEL.ROI_RELATION_HEAD.DPL.RADIUS 1.0
GLOBAL_SETTING.DATASET_CHOICE "VG"
for “MODEL.ROI_RELATION_HEAD.DPL.FREQ_BASED_DIFF_N”, set to False.

Then, the results of the experiment were:
the log of training，The log file is uploaded as an attachment.

SGG eval: R @ 20: 0.5636; R @ 50: 0.6286; R @ 100: 0.6449; for mode=predcls, type=Recall(Main).
SGG eval: ngR @ 20: 0.6417; ngR @ 50: 0.7799; ngR @ 100: 0.8490; for mode=predcls, type=No Graph Constraint Recall(Main).
SGG eval: zR @ 20: 0.0096; zR @ 50: 0.0170; zR @ 100: 0.0222; for mode=predcls, type=Zero Shot Recall.
SGG eval: mR @ 20: 0.1869; mR @ 50: 0.2405; mR @ 100: 0.2708; for mode=predcls, type=Mean Recall.

I also run the test.sh:

export gpu_num=1
export CUDA_VISIBLE_DEVICES=2
checkpoint_dir="xxx/checkpoints/VG/vctree/predcls/DPL"
python3 tools/relation_test_net.py
--config-file "$checkpoint_dir/config.yml"
TEST.IMS_PER_BATCH 1
MODEL.ROI_RELATION_HEAD.EVALUATE_REL_PROPOSAL False
TEST.ALLOW_LOAD_FROM_CACHE True \

the result is：

SGG eval: R @ 20: 0.2718; R @ 50: 0.3738; R @ 100: 0.4333; for mode=predcls, type=Recall(Main).
SGG eval: ngR @ 20: 0.2887; ngR @ 50: 0.4189; ngR @ 100: 0.5199; for mode=predcls, type=No Graph Constraint Recall(Main).
SGG eval: zR @ 20: 0.0056; zR @ 50: 0.0105; zR @ 100: 0.0157; for mode=predcls, type=Zero Shot Recall.
SGG eval: mR @ 20: 0.0400; mR @ 50: 0.0637; mR @ 100: 0.0827; for mode=predcls, type=Mean Recall.
log_motif.txt
log_motif.txt

[Zhuzi24]

Looking forward to your reply, thanks!

[JeonJaeHyeong]

I think the directories for train and test are different. The training path is $EXP/VG/motif/predcls/DPL, but for testing, it's "xxx/checkpoints/VG/vctree/predcls/DPL".

If you have any other issues, feel free to ask questions anytime.