[antixk]

I guess OP meant that each tensor core in the latest NVIDIA GPUs fundamentaly performs 4x4 matrix multiplication [0].

[0] https://nla-group.org/2020/07/21/numerical-behaviour-of-tens...

[shaklee3]

That's also not true. They do 4x4, 8x8, and 16x16.

[AnotherGoodName]

Which is fine for the point being made here. There's no issue with n^3 for small matrices.