[ashed96]

In my experience, LLMs tend to take noticeably longer to process images than text.

[weird-eye-issue]

It has to get the image data first, basically just IO time before processing it

[ashed96]

IIRC there's pre-processing (embedding/tokenization?) before feeding images to LLMs?

Hit this issue optimizing LLM request times. Ending up lowering image resolution. Lost some accuracy but could bear that.

[psadri]

I wonder if these stay in the prefix cache?