[ekelsen]

Image patches are projected directly into an embedding that goes into the decoder Transformer. The same thing could be done for audio.