[whoknowsidont]

LINQ is going to add overhead, regardless of "properly" using it or "cargo-culting" things; save the platitudes for the Monday zoom meeting.

LINQ adding overhead is a _technical reality_, it's how it works and that is fine. It's a fine tool in many difference contexts, but when we talk about performant code the context is obviously one in which every cycle matters.

And those of us with enough experience know that LINQ performance and implementation details varies over time in the runtime, and those shifts aren't always positive.

So when writing code where performance is fundamental to the success of the application, avoid LINQ since it WILL add overhead and it will remove implementation control from your team. It is a risk without much benefit when you're in the performance arena. That doesn't mean it's not useful in many other contexts.

[fabian2k]

There are cases where using the straightforward LINQ code would be a lot faster than a lower level alternative. For example when the code can be vectorized and use AVX instructions, which is implemented for quite a few LINQ methods. A straightforward non-LINQ version of the code would likely be slower as most developers would not or can't write the low-level AVX version.

I'd certainly be careful about LINQ in certain performance-sensitive code, e.g. about creating unnecessary copies of the data and allocating too much. But I would not trust myself without measuring to really know whether it actually makes a difference or if my "optimized" code might be even slower.

[progmetaldev]

A lot of LINQ was also optimized and improved with the move to .NET Core (now just .NET). It's definitely important to actually profile the code, rather than just assume LINQ is slow/less-performant. Most of the time, unless the developer has added unneeded code (such as calling .AsEnumerable, or most anything that evaluates the entire collection), the difference between LINQ and standard iterator based code will be nearly non-existent, with some of the cases you mentioned where LINQ has been optimized beyond what a developer can do by hand.

[Const-me]

> AVX instructions, which is implemented for quite a few LINQ methods

Are you sure? Any examples of such methods? And does AVX actually helps?

I don’t think that’s possible because IMO AVX and other SIMD can only help for dense inputs. The C# type is ReadOnlySpan, however ReadOnlySpan doesn’t implement IEnumerable and therefore incompatible with LINQ.

There’s even an alternative LINQ to workaround https://github.com/NetFabric/NetFabric.Hyperlinq but that thing is a third-party library most people aren’t using.

[neonsunset]

- https://github.com/dotnet/runtime/blob/main/src/libraries/Sy...

- https://github.com/dotnet/runtime/blob/main/src/libraries/Sy...

- https://github.com/dotnet/runtime/blob/main/src/libraries/Sy...

[Const-me]

Interesting, I never heard about that. Was merged to master in February 2022, I wonder which release version includes the changes?

Still, the support seems very limited. They simply probe argument type for arrays and lists. Any other IEnumerable gonna return false from TryGetSpan, which reverts to the legacy scalar implementation.

[merb]

pretty sure thats dotnet 7 only stuff. I can vaguely remember a blog post about it: https://devblogs.microsoft.com/dotnet/performance_improvemen... (I think this is the one?)

it's faster in bigger arrays/lists but smaller ones barely make a difference, even the linq vs non-linq make basically only noise difference as far as I remember.

[neonsunset]

Algorithms that work on non-contiguous or synthesized data are not subject to vectorization in any language, aside from select cases where LLVM is able to inline and auto-vectorize loops in Rust for certain simple cases of map->collect.

What is your argument then?

[Const-me]

My point is the support is very limited, only works for a few trivial use cases, like sum/min/max/average of arrays of a few selected types.

I believe it’s technically possible to vectorize more complicated stuff in C#, just the runtime library is not doing that. For an example, look at how Eigen C++ library https://eigen.tuxfamily.org/index.php?title=Main_Page does their math. Under the hood, they wrap inputs into classes which supply SIMD registers, then do math on these registers. Eigen does that in compile-time with template metaprogramming. A hypothetical C# implementation could do similar things using generics and/or runtime code generation. LINQ from the standard library was never designed for high-performance compute, but I think it might be possible to design similar API for that.