AttributeError: 'TransformerDecoderLayerOptimal' object has no attribute 'self_attn'

[agathebeucher]

When i try to train the model on MSCOCO2014, i have this error that i can't fix : AttributeError: 'TransformerDecoderLayerOptimal' object has no attribute 'self_attn'... what should i do ?

[DatesYuen]

I have meet the same question.

[DatesYuen]

It has been resolved when I change Pytorch version from 2.1.1 to 1.13.1.

[agathebeucher]

But when i do that i have the error : "ModuleNotFoundError : No module named 'torch._dynamo'...

[agathebeucher]

Did you solve that ?

[DatesYuen]

But when i do that i have the error : "ModuleNotFoundError : No module named 'torch._dynamo'...

I don't have that problem. I can run this project after change Pytorch version.

[fnauman]

Found a fix. Use vanilla nn.TransformerDecoderLayer instead of the custom TransformerDecoderLayerOptimal in Line 124 of ml_decoder.py:

ML_Decoder/src_files/ml_decoder/ml_decoder.py

Line 124 in 8a9e984

layer_decode = TransformerDecoderLayerOptimal(d_model=decoder_embedding,

layer_decode = nn.TransformerDecoderLayer(
    d_model=decoder_embedding, 
    nhead=8, 
    dim_feedforward=dim_feedforward, 
    dropout=decoder_dropout
)

Did you solve that ?

[Esmaeel-Hilal]

Install PyTorch 1.7 to solve the issue