are attention_masks zero tensors?

[GinnyXiao]

Dear authors,

I was wondering during draining what attention masks were you using? In your inference code I saw you generated these masks by setting them to zero tensors text_padding_position=torch.zeros_like(input_ids). Was it the same for training? Thank you so much!

def forward(...):
    ...
    output = self.mm_extractor.beit3(
            visual_tokens=images_evf, 
            textual_tokens=input_ids, 
            text_padding_position=~attention_masks
            )
    ...

[CoderZhangYx]

We place no attention at padding tokens. Our dataset code may produce the attention_mask. Simply using that is okay.

[GinnyXiao]

Got it, thank you soooo much!