[tholor]

Yep, you want a GPU for many of the models. However, then subsecond queries are easy to achieve. See benchmarks: https://haystack.deepset.ai/benchmarks/latest