Hacker News new | ask | show | jobs
by cm2187 2692 days ago
Though my understanding is that if you do something like this:

var a = new double[10000];

var b = a.Select(x=>x*x).ToArray();

I'd expect LINQ to create a List or a similar structure when executing the select, then copy the output to the final array. If you were operating on two arrays directly, I would expect less memory allocations (unless these things get optimized away).

1 comments

No. The Select-statement is not evaluated until necessary, i.e. until ToArray() is called.
While your comment is true, I'm not sure how it's relevant. `ToArray` is still operating on the `IEnumerable` produced by Select, which doesn't tell you that the Select is itself operating on an array.

The CLR does do some type checking to try to propagate the size if it can, but that's by no means guaranteed.

Edit: In skimming the source available on sourceof.net, it looks to me like the array doesn't actually get propagated far enough, so the construct in GP will in fact incur several unnecessary array copies.

I'm curious what you saw. My understanding was that it's mostly generators, i.e. it's done through yields, not extra allocated arrays.
It's `ToArray` itself that's the problem. Since it doesn't know how big the array will be in the first place, it has to allocate a small array, start iterating, copy everything into a bigger array each time it runs out of space, and finally allocate an array of exactly the right size and copy everything into that.

You can see the `ToArray` implementation at [0], which defers the implementation to [1]. The implementation checks for ICollection to get the exact size, but the type [2] that `Select` returns doesn't implement ICollection, so `ToArray` has to fall back to the less efficient algorithm.

[0] https://referencesource.microsoft.com/#System.Core/System/Li...

[1] https://referencesource.microsoft.com/#System.Core/System/Li...

[2] https://referencesource.microsoft.com/#System.Core/System/Li...

That's more or less what I had in mind without being certain of the details.

Was trying to open your links but I am served an expired tls certificate from Microsoft!

Oddly enough it seems that the certificate just happened to expire today in between the posting of my first and second comment. I didn't mention it only because I'm at work and couldn't be 100% certain that it's not my organizations security service messing with things.