[chrismorgan]

> In React, we render lists by using regular JavaScript idioms like loops, arrays, and array methods like map. However in Solid.js, much like traditional templating languages, we get a construct like <For> that reinvents a concept that's already in the language.

Once you deal with larger amounts of data and need virtualised rather than fully-materialised lists, you start using different things in React as well. The fact of the matter is that if you care about performance at all, the simple ways are just insufficient, and the native language constructs were designed for procedural programming, not reactive interface rendering, which requires fundamentally incompatible semantics. It’s not even fair to claim that React uses regular JavaScript idioms—VDOM, hooks, the entire shebang is all about eschewing regular JavaScript idioms because they don’t scale. (OK, so there’s also the matter of transient state like scroll positions, element focus, and form field values; it’s not fair to say that React does all these things purely for performance’s sake, as the naive immediate mode approach would also break such functionality.)

[mwcampbell]

> virtualised rather than fully-materialised lists

I want to push back somewhat on this practice. Our computers are fast enough now, and the browser implementations optimized enough, that they should be able to handle thousands of materialized list items without breaking a sweat. Sometimes you really need virtualization, e.g. if the underlying data source has millions of records. But if the data can be fully materialized, then the implementation is simpler, and the user can take advantage of things like find in page. Virtualization is a convenient way to avoid the inefficiency of unoptimized VDOM-based rendering (e.g. with React, and yes, I know there are other optimizations available in React), but fine-grained updating (as in Solid) is even better.

[chrismorgan]

A few hundred, sure; a few thousand, it’s starting to get a bit iffy.

It depends a little on the complexity of the rendering for each item, and where you are fetching the data from, but when you’re into thousands of records you’re very likely to need at least some partial rendering. Suppose each record’s data is one kilobyte (I’ve seen far lower and far higher), then one thousand records is already one megabyte, which for many people will take multiple seconds to transfer, so you’ll still want to load the visible records before fetching more, or do streaming parsing of the records as they come in. And that’s ignoring the backend’s performance on fetching records, which must be taken into account too.

People are certainly often too eager to reach for virtualised lists, or worse still lazy loading without reserved scroll height, but even at a thousand records with simple rendering they’re still probably generally warranted—fast computers can cope with comparatively little visible difference, but on slower ones (especially older and cheaper phones) you’ll easily feel the difference. Memory usage can also be a concern for larger quantities of data and DOM (1000 × 100KB = 100MB).

I’m saying all this as one that scorns React and VDOM stuff in general as unnecessary performance overhead, and likes to use Svelte and plain JavaScript and things like that (or better still, to eschew JavaScript); and who worked on Fastmail’s webmail, which certainly uses such progressive-loading lists, on a precise-DOM-updates framework that cares significantly about runtime performance. React and its ilk are certainly particularly prone to using virtualised lists as a crutch to work around their shortcomings.

All that being said: yeah, I wish things like Discourse would stop doing aggressively lazy loading when there aren’t even several hundred comments in a thread. Render just the things on screen to begin with, if you must (though it’d be better to just send real HTML from the server and let the browser take care of all this, even if it complicates your JavaScript loading), but then load all the rest straight away so that I’m not penalised just because I’m on the other side of the world from the server, and let the browser handle in-page search.

[mwcampbell]

Aggressive virtualization, especially if it also involves removing stuff that scrolled out of view, also bogs down some screen readers (particularly Windows ones) that have their own representation of the web page.

Speaking of both Discourse and screen readers, before my stint at Microsoft, I wrote a Windows screen reader, which tried to detect client-side page navigation by watching for the URL (minus the fragment) to change. Discourse's infinite scrolling implementation broke this heuristic, because Discourse would use the history API to update the URL as the user scrolled. Not sure if I or they were in the wrong there.

Interesting that you feel that Discourse penalizes you for being far away from the origin server. When Discourse was new, one of the founders blogged about how their heavy use of client-side JavaScript made the application better for users far away from the origin server:

https://eviltrout.com/2013/01/06/turbolinks-and-the-prague-e...

Maybe the author had a point, but it sounds like Discourse still relies too much on frequent round trips.

Edit to add:

> a precise-DOM-updates framework that cares significantly about runtime performance

That's Overture, right? I wonder how it compares to Svelte and Solid.

[suction]

What JS framework would you choose to work with big datasets like, e.g. a data grid with half a million rows that should have a "filter as you type" functionality?

[chrismorgan]

Most of the work for this is essentially database tech rather than UI tech. From the UI perspective, you just need to be able to say things like “current query is ‘foo’, and based on my current scroll position I want to render records 32–86” (since 42–76 will be visible on screen, and then we add a few more for good measure to give a small time buffer for retrieving more when you start scrolling), ask the database layer for the required records, and render them as a perfectly normal virtualised/only-partially-materialised list. It’s then up to your database layer to perform the filtering; whether that database runs on the frontend or backend makes no difference, and whether it’s sqlite.js or records.filter(…).slice(…) makes no difference (though their performance characteristics will certainly vary). This can be integrated with the UI framework fairly tightly, but there’s no need for it to be.

For the UI part of it: what I would use would depend on my requirements (is it a list, is it a grid, how is it to be interacted with, &c.) and what was already in use (React, Svelte, plain JavaScript, other). I personally would often be inclined to implement it from scratch, because I’m typically not impressed with most library options (they have a tendency to be heavy, limited, and slower than they need to be) and am familiar with exactly what needs to go into it to make it as perfect as is possible (it’s not a particularly large amount of work, but it is fiddly in places and must be done correctly or it’ll be awful), but that’s not a course of action I would recommend for most developers.

[littlecranky67]

Doesn't matter which framework if you implement it right. Filtering data from the dataset is plain js, and it will be slow. Rendering the data to screen requires a lot of work, such as using offscreen-buffers for smooth scrolling etc. Actually ag-grid does this pretty well, and I've used it for similar in a React app. Now hold your horses, ag-grid is angular 1.x, encapsulated as a component. Another proof that the framework is not important, you need to optimize hell out of it.

[victorNicollet]

Having recently shopped around, and implemented, this kind of data grid: no JS framework actually handles the hard parts, but (more-or-less complete) libraries exist for nearly all of them.

My specific use case is a React application, and I have found it easier to implement a dedicated listener system, than try to fit things into component states.

[PetahNZ]

Isn't that what a backend is for?

[pygy_]

You shouldn’t do that kind of work from the rendering thread, and you should use occlusion culling to only render the needed DOM

[suction]

Hence my question, is there a datagrid for any JS framework that does all the hard stuff for me?