Hacker News new | ask | show | jobs
by mwcampbell 1566 days ago
> virtualised rather than fully-materialised lists

I want to push back somewhat on this practice. Our computers are fast enough now, and the browser implementations optimized enough, that they should be able to handle thousands of materialized list items without breaking a sweat. Sometimes you really need virtualization, e.g. if the underlying data source has millions of records. But if the data can be fully materialized, then the implementation is simpler, and the user can take advantage of things like find in page. Virtualization is a convenient way to avoid the inefficiency of unoptimized VDOM-based rendering (e.g. with React, and yes, I know there are other optimizations available in React), but fine-grained updating (as in Solid) is even better.

1 comments

A few hundred, sure; a few thousand, it’s starting to get a bit iffy.

It depends a little on the complexity of the rendering for each item, and where you are fetching the data from, but when you’re into thousands of records you’re very likely to need at least some partial rendering. Suppose each record’s data is one kilobyte (I’ve seen far lower and far higher), then one thousand records is already one megabyte, which for many people will take multiple seconds to transfer, so you’ll still want to load the visible records before fetching more, or do streaming parsing of the records as they come in. And that’s ignoring the backend’s performance on fetching records, which must be taken into account too.

People are certainly often too eager to reach for virtualised lists, or worse still lazy loading without reserved scroll height, but even at a thousand records with simple rendering they’re still probably generally warranted—fast computers can cope with comparatively little visible difference, but on slower ones (especially older and cheaper phones) you’ll easily feel the difference. Memory usage can also be a concern for larger quantities of data and DOM (1000 × 100KB = 100MB).

I’m saying all this as one that scorns React and VDOM stuff in general as unnecessary performance overhead, and likes to use Svelte and plain JavaScript and things like that (or better still, to eschew JavaScript); and who worked on Fastmail’s webmail, which certainly uses such progressive-loading lists, on a precise-DOM-updates framework that cares significantly about runtime performance. React and its ilk are certainly particularly prone to using virtualised lists as a crutch to work around their shortcomings.

All that being said: yeah, I wish things like Discourse would stop doing aggressively lazy loading when there aren’t even several hundred comments in a thread. Render just the things on screen to begin with, if you must (though it’d be better to just send real HTML from the server and let the browser take care of all this, even if it complicates your JavaScript loading), but then load all the rest straight away so that I’m not penalised just because I’m on the other side of the world from the server, and let the browser handle in-page search.

Aggressive virtualization, especially if it also involves removing stuff that scrolled out of view, also bogs down some screen readers (particularly Windows ones) that have their own representation of the web page.

Speaking of both Discourse and screen readers, before my stint at Microsoft, I wrote a Windows screen reader, which tried to detect client-side page navigation by watching for the URL (minus the fragment) to change. Discourse's infinite scrolling implementation broke this heuristic, because Discourse would use the history API to update the URL as the user scrolled. Not sure if I or they were in the wrong there.

Interesting that you feel that Discourse penalizes you for being far away from the origin server. When Discourse was new, one of the founders blogged about how their heavy use of client-side JavaScript made the application better for users far away from the origin server:

https://eviltrout.com/2013/01/06/turbolinks-and-the-prague-e...

Maybe the author had a point, but it sounds like Discourse still relies too much on frequent round trips.

Edit to add:

> a precise-DOM-updates framework that cares significantly about runtime performance

That's Overture, right? I wonder how it compares to Svelte and Solid.