From assignment 1 the shape of Q, K and the formula (11) does not give a valid matmul 🤔

[riochuong]

[Aprsun]

I think they have confused matrices with vectors. For the multiplication of two matrices, as long as the dimensions are determined, the i,j-th value is always obtained by multiplying the i-th row vector by the j-th column vector, whether in implementation or mathematically. There is nothing about row vectors or column vectors. Therefore, I believe it should still be QK^T here.

[marcelroed]

Yes, this is a typo and will be fixed for this year's release. I'll leave the issue open until we fix it.