[mlthoughts2018]

I’d go further and say that deep networks are excellent for sentence understanding, and various types of RNN or 1D convolutional layers are very good at this in specialized domains just as CNNs and ResNets are good in specialized vision applications.

It absolutely makes sense to use deep learning for both of these tasks.

In fact, one very effective thing to do is to use a Siamese network to learn joint representational spaces of text and imagery in the same network.

It’s really specious and disingenuous to say “boy, vision and language sure seem different but can you believe these DL researchers are using the same tools for both!?”

[mattmcknight]

or... "vision and language sure seem different, can you believe that networks of neurons in the brain do both?"