[whalesalad]

I don't really know how to interpret this comment. 3.2M rows are not loading into the browser at once, only about 10k. The page size is configurable. The frontend and backend have a contract to agree on how this works, so as the user scrolls (and frontend needs another page) it asks the backend for more. The frontend will keep up to N pages (also configurable) cached in the client. Works shockingly well without any fuss at all. The AG-Grid team really did a great job here (it is not free for this particular feature, but well worth the cost)

[taeric]

Apologies, it was not the technical side of it that had me musing on it. I was genuinely curious why and how I would "scroll" through that much data in a meaningful way.

And I don't mean this as a heavy criticism of the idea. I'm assuming it is useful to you. Always fun to hear about how this sort of thing is used.

[warpspin]

You don't scroll through such a list - you use filters, pivoting etc. For end users, it's often quite comfortable to unify this in an Excel-like interface, as they're used to it. In most CRUD projects I had, users had an aversion against paged/pre-filtered displays and rather would have everything in one list where they can dynamically filter it if necessary.

[taeric]

Right. But that just brings us back to "how do people work with a scrolling list of 3.2m records?"

I get the point of wanting it locally to use power tools. And I get that the browser is probably capable of implementing a lot of power tools. Seems silly to insist on doing it all "in memory" on the browser, though?

That is, if the idea is you are doing pivots and filters, I don't know why a server side hit wouldn't be better for that. Similarly, when I look at something like a stock ticker for the day, I don't expect every single transaction was sent to my browser to create the graph. It /could/ be done that way, but why?

More directly to the question I had here, why and how would someone need a scroll list of every market transaction? For fine audits, I would get it, but even then I'd expect some sort of search or anomaly detection?

Still, I think if the answer is to "get it in the users hand and let them do what they will with the data," I can accept that. Goal isn't necessarily to let the users scroll the data endlessly, but for them to use any bespoke tooling they are already using.