ORMs are just a tool, and they really don't preclude query optimisation.
All the ones I've used or written allow bypassing selects, joins or an entire query and manually translating results, so they don't have to get in the way when optimisation really matters, but the vast majority of the time IME in most apps that just isn't necessary so I'll take the reduced friction of a query builder that automates the basics as long as it allows bypassing it when required.
I guess my problem isn't with ORM's, its with ORM SQL generation.