I have had a number of contracts where my entire job was to fix the performance problems introduced by using a ORM. ORMs add a layer of complexity to your code and a huge risk of really bad SQL.
ORMs can be used poorly sure, but anything I can design using raw SQL I can typically do with an ORM. The question is, are people using ORMs as a way to avoid having to figure out proper database design? If so, they should not be using an ORM and should really learn how the database they will be using works. Heck! There's micro ORMs like Dapper which let you run raw SQL and match it to an actual object for you, to be fair almost all ORM's support doing raw SQL and getting back an object which can be handy for performance bottlenecks. Badly coded software will usually have performance issues, whether using raw SQL or not.
Do you think the performance would have been better if the same programmers who introduced them using the ORM had been forced to use raw SQL?
I've seen far too many cases where programmers get around their lack of SQL knowledge by writing a bunch of basic queries and then post-processing the results client side.
Update: Fixed typos.