when i use Animatediff?RuntimeError: mat1 and mat2 shapes cannot be multiplied (2480x2048 and 768x320)

[anthonyyuan]

key = attn.to_k(encoder_hidden_states)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/anthony/miniconda3/envs/p12/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/anthony/miniconda3/envs/p12/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/anthony/miniconda3/envs/p12/lib/python3.11/site-packages/peft/tuners/lora/layer.py", line 495, in forward
result = self.base_layer(x, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/anthony/miniconda3/envs/p12/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/anthony/miniconda3/envs/p12/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/anthony/miniconda3/envs/p12/lib/python3.11/site-packages/torch/nn/modules/linear.py", line 116, in forward
return F.linear(input, self.weight, self.bias)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: mat1 and mat2 shapes cannot be multiplied (2480x2048 and 768x320)

[budui]

Can you share the code you used? It seems that the reason is that you directly gave the T5 features instead of the ella-converted features to Animatediff.