Hacker News new | ask | show | jobs
by Analemma_ 2692 days ago
The thing about Linq efficiency is that under the hood it’s setting up a state machine with enumerators and continuations to be fully general. It’s about as fully-optimized as it could possibly be given the design constraints, but it will always be less efficient than just iterating over a collection directly.

But the worst performance hits in Linq come when people don’t know how to use it. The biggest culprit is unnecessary materializations, e.g. ToList() or anything that requires traversing the entire enumerator like All(), so if you’re careful to avoid the obvious pitfalls you should be fine.

1 comments

That's... not really true of LINQ, specifically. I have seen this assumption from developers before, with regard to LINQ-to-objects, that 'the compiler does clever stuff to optimize LINQ expressions', but... no, not really. It compiles a series of method calls. The JIT might do some optimization. But it's not a query optimizer - it just does what you tell it, and underneath it all there's nothing but an enumerator.MoveNext() getting called repeatedly.

Setting up state machines and continuations is something C# does when you write generators (with yield) and async methods (with await), but LINQ doesn't do that, particularly.

LINQ the language feature in particular doesn't do any of that sort of thing. As a language feature, LINQ is just the 'query comprehension' syntax, which is the 'from x in y where bar select z' coding form. And from the language's point of view that is just a different way of writing y.Where(x => bar).Select(x => z), which it will then try to compile. If it happens to wind up calling generator methods which implement state machines and continuations, or just simple methods that return ordinary objects, LINQ doesn't care, so long as the types all line up.

The only other thing that confuses matters with LINQ is that those lambdas it wants to compile can match method signatures that have delegate types, or Expression<> types, in which case the lambda doesn't get compiled as executable code, but rather as an expression tree literal, which is how Linq to SQL and friends were built (but again, that's not a 'LINQ' feature - you can assign or pass a lambda as an Expression<> yourself without involving 'LINQ'). And THAT is where LINQ gets a lot of its worst reputation, because that turns out to introduce a ton of complexity and failure modes that aren't the fault of the C# language, except in as much as it made it possible for a library to get involved in interpreting your code's syntax tree at runtime, which may have been a mistake.