[corresation]

That problem is not unique to LINQ or .NET.

It absolutely isn't unique to it -- powerful features allow programmers to shoot themselves in the feet, and this is true since we began this profession.

When dealing with larger applications and datasets it has the potential of easily imposing enormous costs, of the sort that absolutely eclipse most other anti-patterns. Appending on an immutable string through recursive function calls suddenly looks like child's play.

The truth is that in most apps, these inefficiencies simply don't matter, and to all of the "just learn how to use it right!" replies, I would say that odds favor that those people aren't using it right at all. But in the end it just doesn't matter that much because small amounts of data, low demands, etc, just makes it an irrelevant cost. Most apps are running with an excess of CPU resources relative to the demands on the app, so it just doesn't matter.

Many .NET developers have never run Visual Studio's profiler. They don't even know what the costs are, but we're in a world where 1500ms to load a basic form is okay.

When you deal with very large scale financial data, as I do, however, your world is a very different world. You don't just have a scrum where you try to teach people that LINQ isn't magic (which of course we did). The risk factors become much higher.

[louthy]

If the milliseconds matter so much then why aren't you using code reviews or unit tests to catch the problem queries?

LINQ isn't just about hitting a database either, it enables a more declarative / functional approach to development in general, which brings huge benefits in terms of parallelisation / robustness / testability.

It seems absurd that you would block your team from using it.

Edit: Oh and btw, you're not the only person who has to deal with "large scale data". Many of us do and manage just fine with LINQ.

[corresation]

Who said LINQ was just about the database? Of particular concern, actually, are LINQ to Objects.

And yes, those milliseconds matter. And it's very strange that I noted that after repeatedly encountering performance issues with LINQ, you say that we should have oversight. Do you understand that such is oversight? That the rules and exceptions came about because the recurring issue with LINQ (particularly LINQ to Objects) being an enormous performance issue?

It doesn't matter to you, probably because you don't even know what the cost is [and that isn't meant to be at all a slam. For most software the cost simply doesn't matter]. And that's fine. But save the continual exclamatives (which you always add right after defensively down-arrowing).

Though I have to chuckle at the notion that LINQ helps with testability, or robustness for that matter.

[louthy]

> That the rules and exceptions came about because the recurring issue with LINQ (particularly LINQ to Objects) being an enormous performance issue?

No, you're exaggerating your position to try and justify it. There is zero substance in what you're saying. All I see is someone who appears to have made an irrational decision about a technology because he personally doesn't like it. There's no fundamental reason why LINQ to Objects should be any slower than say a foreach over a set. You can get is a mess with either approach. If you're not prepared to mentor your team, or do code reviews to check for problematic usage then it's a fault of yours, not the technology.

> It doesn't matter to you, probably because you don't even know what the cost is [and that isn't meant to be at all a slam. For most software the cost simply doesn't matter].

Of course it's meant to be a slam, you have no knowledge of my experience of optimisation or my deeper understanding of how frameworks like LINQ works, so instead you assert that I "don't even know what the cost is". I have spent a large part of my career doing 'to the metal' optimisation, so I will ignore that comment.

> Though I have to chuckle at the notion that LINQ helps with testability, or robustness for that matter.

If you don't understand why imperative and functional code are different then you won't understand why LINQ is inherently more robust and testable.

[corresation]

There's no fundamental reason why LINQ to Objects should be any slower than say a foreach over a set.

There's no fundamental reason why LINQ to Objects should be any faster than a foreach over a set. Which is precisely the point. LINQ has an amazing way of hiding details in a manner that allows for massively inefficient code to look completely innocuous. For throw-away, one-off "command line utility to convert A to B" type code, it is absolutely golden. For long-term use code that may be called millions of times, it can be disastrous.

You have no interest in a reasoned discussion, and your argument style is best described as defensive. That is all. Have a nice day.

[louthy]

This is quite incredible. Maybe have a read through the various arguments you're having on this entire topic and see who's being defensive. If you really cared about performance you wouldn't use C#, you'd use C or C++. Anything where you could absolutely control every aspect of memory allocation, memory access, bounds checking, IO, etc.

I'm out of this thread, it's not adding to the discussion.

[corresation]

If you really cared about performance you wouldn't use C#, you'd use C or C++

This, in a nutshell, is your argument. And it's an incredibly poor, out of place argument that is founded on ignorance, and does absolutely no service to .NET.

Performance in most modern applications is achieved through proper algorithms and data structures. LINQ is often a hasty short-cut around those, but it offers the illusion of elegance such that many people (such as, apparently, you. It's all functional-like and such, so it has to be good, right?), are blissfully unaware.

Again, a lot of the time that's perfectly fine and causes no real harm -- similar to the way regular expressions might be a costly but powerful catch all -- but in other cases it is not good. That is the case with large scale financial software. If you have a different experience, good for, but simply saying "incredible!" over and over again does absolutely nothing to make your case.