invalid argument 0: Sizes of tensors must match except in dimension 0. Got 27 and 41 in dimension

[tuyunbin]

Hi, I met a bug, but I haven't fixed it. I want to use this code to train model on the MSVD dataset. I can train the model on the MSR-VTT. So I prepare the caption.json and info.json based on default setting(like MSR-VTT), but when I train the model on the MSVD, I met the problem:

Traceback (most recent call last):
File "/home/tuyunbin/video-caption.pytorch/train.py", line 139, in
main(opt)
File "/home/tuyunbin/video-caption.pytorch/train.py", line 124, in main
train(dataloader, model, crit, optimizer, exp_lr_scheduler, opt, rl_crit)
File "/home/tuyunbin/video-caption.pytorch/train.py", line 33, in train
for data in loader:
File "/home/tuyunbin/anaconda3/envs/caffe2/lib/python2.7/site-packages/torch/utils/data/dataloader.py", line 560, in next
batch = self.collate_fn([self.dataset[i] for i in indices])
File "/home/tuyunbin/anaconda3/envs/caffe2/lib/python2.7/site-packages/torch/utils/data/_utils/collate.py", line 63, in default_collate
return {key: default_collate([d[key] for d in batch]) for key in batch[0]}
File "/home/tuyunbin/anaconda3/envs/caffe2/lib/python2.7/site-packages/torch/utils/data/_utils/collate.py", line 63, in
return {key: default_collate([d[key] for d in batch]) for key in batch[0]}
File "/home/tuyunbin/anaconda3/envs/caffe2/lib/python2.7/site-packages/torch/utils/data/_utils/collate.py", line 43, in default_collate
return torch.stack(batch, 0, out=out)
RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0. Got 31 and 44 in dimension 1 at /pytorch/aten/src/TH/generic/THTensor.cpp:711

When I set the batch_size =1, the code of train.py could run. But when I change this value, the error will occur.

[holibert]

#41 the same as it.
You must make sure one idx, one video id, one caption. Otherwise, the dismatch error will occur.