[bzbarsky]

Ah, but it does result in stagnation on many axes. Where's the Google or Apple effort to add significant intra-page parallelism to WebKit? It's too hard in that codebase, so it's not happening. If you require that codebase for a web browser, that means no significant parallelism in web browsers.

(There's some work being done to parallelize the rendering pipeline, and a bit on painting, but parallel CSS layout seems to be completely off the table in existing browser implementations.)

[malandrew]

Since you know a lot about WebKit and parallelism, especially intra-page parallelism, I'd really like to know a lot more about the problems with it. Can you give a brief rundown of the issues and the design decisions they made (and the correct decision IYHO) and maybe some links on where I can find more info?

[bzbarsky]

So a caveat: I don't feel like I know a lot about WebKit. I know a bit about WebKit. I do know a lot about Gecko. ;)

The issues that I've seen are basically endemic to every large C++ application I've seen, actually: shared memory being accessed via pointers from many different places, lazily computed values, shared global (or even just per-page in the case of intra-page parallelism) caches.

These are all reasonable things to do, so I'm hesitant to call them "incorrect", but if you want to design with parallelism in mind you have to either avoid them or carefully design around having low contention on shared resources, access to mutable shared resources protected by some sort of serialization mechanism, etc.

As a simple example, last I looked WebKit's CSS selector matching algorithm cached some state on nodes as it went (as does Gecko's). That means that as things stand you can't do selector matching on multiple nodes in parallel because you can get data races. This one instance can be worked around by deferring the setting of the cached bits until a serialization point, but you have to make sure you do this for every single algorithm that touches non-const members on nodes that you want to parallelize... And C++ doesn't really give you a good way to find all such, given "mutable" and const_cast.

Another example: When WebKit does layout-related things, it directly touches the DOM from the layout code, which is a perfectly reasonable thing to do when you think about it. But it does mean that it can't do layout in parallel with running JS that can modify the DOM. For that you need to have layout operating on an immutable snapshot of the DOM.

As far as more info.... https://github.com/mozilla/servo/wiki/Design is perhaps a start for what we think we know about how this _ought_ to work. Maybe. ;)

[malandrew]

So earlier I said that making another rendering engine would be a 5-7+ project. How about taking WebKit, forking it and making it highly parallelizable, piece by piece? Is that even possible, or is a big rewrite from scratch with parallelism in mind the only way to go?

[bzbarsky]

We certainly considered that path, with both Gecko and WebKit.

In theory, that's possible, but in practice it would likely take just as long, and at the end you would have a rendering engine that was years behind the competition. Not least because intermediate stages wouldn't be upliftable back to the main codebase because partially parallelizing things leaves you with worse performance than either parallelizing nothing or everything, in many cases.