I would say that LINQ is one of the worst things to ever happen to .NET. On one team I had to make a dictum that barred LINQ from even being used, except in exceptional circumstances.
I do not mean to raise the ire of defensive .NET programmers by saying this: as with most things, used right it is a beautiful thing, and there is nothing fundamentally wrong with LINQ. But what it was used for in practice was almost always grotesquely inefficient.
If someone had to make a loop to iterate objects to find a member on every function call, pretty soon they'll realize they should using a System.Collections.Generic.Dictionary for instance, or some other appropriate data structure (e.g. Need sorted data? Then use a sorted tree, for instance). When it's a simple LINQ query, though, it's one seemingly harmless little line, so it tends to get a pass.
Pretty soon profiles were just hundreds (if not thousands) of different cases where LINQ was used as a cure-all, destroying performance under any load at all.
LINQ is not magical. It is doing the same dirty work that you would be doing yourself if you needed to filter / sort / recast, but provides syntactical sugar to do so. The compiler and runtime will happily generate horrifically inefficient code if you request it, with no warning or complaints.
The tldr; is that it is a fantastic set abstraction that is more often than not grossly abused.
I would change this a little bit. LINQ creates much more readable code, which is always a good thing.
I think the real problem is Cargo Cult programming. People are using linq (especially in combination with EF) and not understanding what's actually going on. So they end up getting things like n+1 selects on collections with thousands of items. They don't understand delayed evaluation. Or how sorting effects a Where() call.
The problem is programmers who think they can take some code from stack overflow and use it without outstanding it. That problem is not unique to LINQ or .NET.
It absolutely isn't unique to it -- powerful features allow programmers to shoot themselves in the feet, and this is true since we began this profession.
When dealing with larger applications and datasets it has the potential of easily imposing enormous costs, of the sort that absolutely eclipse most other anti-patterns. Appending on an immutable string through recursive function calls suddenly looks like child's play.
The truth is that in most apps, these inefficiencies simply don't matter, and to all of the "just learn how to use it right!" replies, I would say that odds favor that those people aren't using it right at all. But in the end it just doesn't matter that much because small amounts of data, low demands, etc, just makes it an irrelevant cost. Most apps are running with an excess of CPU resources relative to the demands on the app, so it just doesn't matter.
Many .NET developers have never run Visual Studio's profiler. They don't even know what the costs are, but we're in a world where 1500ms to load a basic form is okay.
When you deal with very large scale financial data, as I do, however, your world is a very different world. You don't just have a scrum where you try to teach people that LINQ isn't magic (which of course we did). The risk factors become much higher.
If the milliseconds matter so much then why aren't you using code reviews or unit tests to catch the problem queries?
LINQ isn't just about hitting a database either, it enables a more declarative / functional approach to development in general, which brings huge benefits in terms of parallelisation / robustness / testability.
It seems absurd that you would block your team from using it.
Edit: Oh and btw, you're not the only person who has to deal with "large scale data". Many of us do and manage just fine with LINQ.
Who said LINQ was just about the database? Of particular concern, actually, are LINQ to Objects.
And yes, those milliseconds matter. And it's very strange that I noted that after repeatedly encountering performance issues with LINQ, you say that we should have oversight. Do you understand that such is oversight? That the rules and exceptions came about because the recurring issue with LINQ (particularly LINQ to Objects) being an enormous performance issue?
It doesn't matter to you, probably because you don't even know what the cost is [and that isn't meant to be at all a slam. For most software the cost simply doesn't matter]. And that's fine. But save the continual exclamatives (which you always add right after defensively down-arrowing).
Though I have to chuckle at the notion that LINQ helps with testability, or robustness for that matter.
> That the rules and exceptions came about because the recurring issue with LINQ (particularly LINQ to Objects) being an enormous performance issue?
No, you're exaggerating your position to try and justify it. There is zero substance in what you're saying. All I see is someone who appears to have made an irrational decision about a technology because he personally doesn't like it. There's no fundamental reason why LINQ to Objects should be any slower than say a foreach over a set. You can get is a mess with either approach. If you're not prepared to mentor your team, or do code reviews to check for problematic usage then it's a fault of yours, not the technology.
> It doesn't matter to you, probably because you don't even know what the cost is [and that isn't meant to be at all a slam. For most software the cost simply doesn't matter].
Of course it's meant to be a slam, you have no knowledge of my experience of optimisation or my deeper understanding of how frameworks like LINQ works, so instead you assert that I "don't even know what the cost is". I have spent a large part of my career doing 'to the metal' optimisation, so I will ignore that comment.
> Though I have to chuckle at the notion that LINQ helps with testability, or robustness for that matter.
If you don't understand why imperative and functional code are different then you won't understand why LINQ is inherently more robust and testable.
> I would say that LINQ is one of the worst things to ever happen to .NET. On one team I had to make a dictum that barred LINQ from even being used, except in exceptional circumstances.
I find it incredible when I see comments like this. What you should have done is instigated a mentoring programme for the programmers who aren't yet up to speed. LINQ is one of the best things to happen to the language and you just go and kill it.
Counter-argument: maybe by completely banning LINQ you're prematurely optimizing? If you can describe some query or aggregation with LINQ, it may not always be super-fast, but it's also less error-prone than hand-writing equivalent, faster code. It seems like it'd be better to allow the usage of LINQ, at least for the purposes of rapid prototyping and implementing behavioral/unit tests; once you're happy the system works as expected, you can hit it with a performance/memory profiler to identify specific parts of the code which need to be optimized.
Also, check out LinqOptimizer -- it analyzes your LINQ queries and can optimize them for better performance (and it's free!): https://github.com/nessos/LinqOptimizer
Oh you entirely intend it to be offensive, which is fine.
Believe what you want. I will say again that of the three "I can't believe it!" responses thus far, I'll bet zero have ever even run their profiler. Ignorance, as they say, is bliss, and in many cases is perfectly fine.
I meant offense to your team, not you. Even though I have used profilers such as NHProf or Ants quite extensively in .NET, you hardly need to use one to understand the implications of a Linq statement - particularly an IEnumerable based one. If your team doesn't understand simple concepts like iteration or projection (and allocation) then you have an uphill battle to fight regardless of the tech you choose.
Yes but LINQ also provides GroupBy() and ToDictionary() which can be used together to replace Where() or First() in subexpressions and restore performance. Simply banning a useful tool's use altogether just means you weren't any more willing to learn how to use the tool correctly than the programmers who were misusing it.