ValueError: Dimension 1 in both shapes must be equal, but are 7 and 30. Shapes are [30,7] and [30,30].

[Shridharpawar77]

There is problem is training. the dimensions of decoder_outputs and attention_output

[acarr3]

I have this issue. Did you ever resolve it?

[Jonathan-Kris]

I encounter this issue too. It arise when I change the Attention Layer implementation using the official one from Keras (Tf 2.7.0).

Main Problem

The problem happen since the official AttentionLayer implementation return 3D tensor instead of 2D, and this cause the input to ConcatenationLayer to mismatch

How the concatenation layer should be, with the second dimension is None

But if you use the official Keras AttentionLayer the second dimension will be set to a fixed value (the max_text_len, in this case 30)

Using Keras official AttentionLayer Implementation

Plot the model to make it more pleasent to see 😄

Bam! the dreadful error arise

Solution

My solution is to use the original Attention Layer implementation from thushv89 , and this require you to downgrade your Tensorflow version to 1.15.2 (See the custom attention layer requirement here.

Since I'm using google colab to run the notebook, the trick is to copy the AttentionLayer implementation directly to the notebook. Then I run the command %tensorflow_version 1.x to downgrade the Tf version.

Changes & Result

Copy and paste the custom attention layer to your notebook

The model now able to run & learn properly

Hope this trick might help and your model can learn like a charm! 😁

[ccudavidlin]

Excuse me, I have the same problem, but I still can't solve it. I can't import custom Attention Layer properly.
Could you please explain how you do that in details?
Thank you for any help you can provide.