[miklosme]

When AR/VR finally happens, UI developers will have to deal with complexity from a completely different paradigm. For me, React's biggest strength has always been its ability to organize complexity into a manageable order. Combine this with the large pool of developers and extensive ecosystem, I think React will be the go-to tool for AR/VR apps. For this reason, I'm super hyped for R3F.

[tuckerconnelly]

I really don't think React will be the go-to tool for VR; it's based on the DOM and trees of function calls, which are both hierarchical, which necessarily means you have the gorilla-banana problem.

If you have a coffee cup on a table in VR, is that coffee cup a child of the table? How do you move the coffee cup off the table and put it onto another table? Is it now a child of that other table? What about the coffee in the cup? Is that a child of the cup? How do you change properties of the coffee without necessarily accessing the table and the cup?

Developers working on 3D systems have developed much better paradigms than the DOM for dealing with this problem. An Entity-Component-System architecture with "constraints" is the current best solution. In that architecture, you would create a coffee cup "entity" with a mesh "component" with another "constraint" component, constraining that coffee cup to the table (or better yet, mass component acted on by a physics system). Then you can simply remove the constraint component when removing the cup from one table, and re-add the constraint component when adding it to the other table.

Overall, I think web developers are in for some intense learning and paradigm shifts if 3D becomes the norm.

[strbean]

I don't see why an ECS would be incompatible with a DOM tree.

As for the gorilla-banana problem, I would think all objects in a scene would be under the root, with the exception of pieces that make up a thing and rarely separate (wheels on a car, for example).

[tuckerconnelly]

While not incompatible with ECS, the DOM and this renderer go all-in on the javascript event-loop. You would have to write your own run loop, which executes the systems on every frame (ideally creating a DAG and executing in parallel while possible), and leave the event loop behind, with all the niceties like `onClick`, to go full ECS. Otherwise you'll create some Frankenstein monster of part ECS, part event-loop, part declarative React.

Additionally, you can throw OOP in that mix as well, because Three.js has it's own whole OOP-style framework, that you're strapping declarative React on top of with this renderer. Reminds me of Jonathan Blow's talk on the end of civilization via endless layers of abstraction[1].

I really think, when it's ready, a Bevy[2]-style system either native or compiled to WASM with WebGPU will be ideal.

And while I'm airing opinions (forgive me), I think writing shaders now is like SQL 30 years ago. Developers left optimizing difficult--according to them--SQL to database administrators by abstracting it away into ORMs. If history is any indicator, I think we'll be having the same arguments on Hacker News 30 years from now about 3D frameworks vs writing shaders directly as we're having now about ORMs vs writing SQL directly.

[1] https://www.youtube.com/watch?v=pW-SOdj4Kkk

[2] https://bevyengine.org/

[dakom]

It's not that it's incompatible, it's that when the ECS is the primary tool for organization, a DOM tree (or scenegraph) is merely one way of iterating over the entities - not the way.

This provides tons of benefits, so for example you can also decide to iterate over the entites by shader program and gain significant speedups for graphics processing, or maintain components that roughly sort them by their position in world space for physics and culling or lighting, etc.

For a crude analogy, imagine if Document.querySelectorAll() were a zero-cost abstraction, i.e. it ran as fast as iterating over linear memory. In practice this isn't how it turns out with an ECS, but it's much closer and you can get this kind of performance for the "hot path" kind of queries.

To add to the sibling comment, there's another wonderful Rust ECS called shipyard[0] and I helped write a scenegraph for it (which I really need to update, one of these days)[1]

[0] https://github.com/leudz/shipyard

[1] https://github.com/dakom/shipyard-scenegraph

[mlsarecmg]

React is not based on the dom, R3f merely expresses regular threejs which works as an object graph. Three is the usual choice for 3D on the web, if you use it once you'll see that it is also quite natural. There is no conflict between the two and react certainly doesn't change any rules or apis, it just builds a graph, which you would normally form imperatively.

[tuckerconnelly]

React is inspired by the DOM and they split it before 1.0 IIRC, but that misses the forest for the trees. The main issue I had is that React, Three.js, and R3F are all hierarchical/tree-like (what you and Three.js are calling a graph). You can technically yes, build 3D scenes, but anything non-trivial will be very awkward.

Let's say you're building a game where you want a sphere to stick to whatever player you throw it at. How would you do that with a scene graph/OOP model? It'd be awkward, removing objects from one parent and adding them to another. Even more awkward if it's a complex object and you only want a part of that complex object to stick to the player. ECS + a constraint or physics system does a decent job (not perfect) of handling this in a relatively elegant and performant way.

I've used Three.js enough--built my portfolio[1] out of it, and then switched to Babylon when I realized how little I liked Three.js. For the record, I also dislike Babylon.

[1] https://tuckerconnelly.com

[mlsarecmg]

i have yet to encounter something that shouldn't be expressed as a graph. three, babylon, ogl, blender, gltf, cad, games, they're all scene aligned. that doesn't seem to be a conflict since you still use shaders, physics, ecs and so on.

could you go more into detail what you mean when you say "anything non trivial"? is there a real example of something that would not be possible to create in, say, threejs?

[chrisco255]

https://codesandbox.io/embed/simple-physics-demo-with-debug-...

There's some excellent demos of how this works with this library with a full physics engine. Loads great even on my phone.

[rock_hard]

Nothing in R3F prevents you from organizing your scene like that

[machiaweliczny]

AFAIK Unity has scene tree(items on table belong to table for example), so it seems compatible. Usually 3D transforms work on scene subtrees.

[SirHound]

Aren’t nested coordinate systems a quite natural match for a DOM-like tree structure?

[nobbis]

As a counterpoint, I've enjoyed building VR on the web using C++ (with WebAssembly/WebGL/WebXR) and not having to touch the DOM or JS, see: https://twitter.com/nobbis/status/1425266634982248451

Benefits include complete control at frame and pixel level, being cross-platform (same code runs on web, iOS, macOS, Linux), and having access to third-party C/C++ libraries for 3D graphics.

[nightski]

I have not put a lot of time into learning WebAssembly. But isn't WebGL a JavaScript API? Meaning wouldn't you be going from WebAssembly -> JavaScript engine -> WebGL? I was under the impression WebAssembly had no access to the outside world and could only access the relevant JavaScript APIs. But if it is true you can basically do WebAssembly -> native GL then that would be amazing.

[nobbis]

You're correct. WebGL does require extra validation compared to native GL, but it's effectively the same API as OpenGL ES 2.0/3.0 and Emscripten handles the translation from C/C++ for you.

There's some overhead but it's negligible (assuming you're not making overly redundant API calls.)

[shadowgovt]

And that last part is key: for modern high-performance graphics acceleration, the name of the game is "maximum throughput with minimum API interactions."

If your data isn't structured for fast rendering, it doesn't matter much what language you're using; they'll all be too slow.

[rock_hard]

This is how Figma does it

[Jasper_]

emscripten ships a "desktop GL" emulation library [0], which can have quite a bit of overhead. If you want something faster, you can use the native WebGL bindings [1]

[0] https://github.com/emscripten-core/emscripten/blob/main/src/... [1] https://github.com/emscripten-core/emscripten/blob/main/test...

[nobbis]

The "native WebGL bindings" still call into JS.

Desktop GL emulation is just a layer on top OpenGL ES if, for example, you're still using OpenGL's fixed function pipeline (deprecated 13 years ago.)

[dakom]

I wrote an opinionated WebGL wrapper in Rust/WASM here: https://github.com/dakom/awsm-web

The idea is that it does more cacheing of things like uniform locations and such so you do very fast in-memory lookups in WASM without hitting the JS Api as much.

In the future this will be obsolete since WebGPU has a more optimal API to begin with, and Rust/WASM won't need to go through the JS layer due to "interface types"

[Unit520]

WebAssembly has no access to the outside world at the moment, that is correct. It is only able to call (and be called by) JS.

A C++ application compiled via Emscripten ships (a fairly large amount) of JS glue code that exposes all relevant Browser APIs like WebGL, Fetch or other HTML5 stuff to the actual WASM program. As others commented, for WebGL an additional API translation is applied. If the source targets OpenGL ES 2 (or 3 for WebGL 2), this step has almost no overhead however.

[nobbis]

Not that large. The JS glue code for the VR web page referenced above is 48 KB, including WebGL, WebXR, Fetch, etc.

[brailsafe]

This looks really neat

[webgfxdev]

React is one of the worst choices of doing something like that.

The underlying abstraction model of having a tree of components and re-rendering only the parts that have changed between renders doesn’t map to the hardware at all, meaning you’ll waste most of the HW performance just on maintaining the abstraction.

You’ll also get zero benefits from the third-party libraries - there’s nothing in them that can help you with stuff that matters, like minimizing amount of the GPU state transitions for example or minimizing amount of GPU/CPU syncs.

It will be scenegraphs all over again, and the graphics industry has ditched these long ago in favor of simpler models, for good reasons.

Long story short, the happy path in graphics programming is very narrow and fragile, and you typically want to structure your abstraction around it.

[mlsarecmg]

You are arguing against threejs not react. R3f reconciles threejs in the exact way it's getting used, a graph. This ofc is also how blender gltf et al work. If you make a webgl app on the web you most likely use three and all react does is make that a little ordered with some additional benefits when it comes to performance, memory and interop.

[xrobledo84]

What is wrong with scenegraphs? And what is the graphics programming using instead?

[rock_hard]

Totally agree!

We switched https://flux.ai from vanilla ThreeJS to R3F and it’s been a huge productivity gain for the whole team!

Less code, that is more capable and more reusable!

In case anyone is interested, we are hiring: https://coda.io/@flux-ai/flux-jobs

[nobbis]

Is this AR/VR? It looks like 2D circuit schematics.

[rock_hard]

It’s 2D schematics and 2D/3D PCB layout. All powered by r3f though

PCB layouting hasn’t publicly shipped yet though

[hirako2000]

Except that for three.js, it's react introducing complexity rather than improving the organisation of the code. A simple component with defaults look neat, but start building a complex scene and jsx gets in the way.

three.js isn't dom elements updated in js. The state of each object is updated in the scene depending on more than whether they changed.

Where three.js lacks abstraction is a component system, in plain js, to organise application with decent patterns. Most three apps are a big blob or code.

[runawaybottle]

Okay yes, thank you from saving me from my drivel. Why would three.js care about representing a document model, bubbling up events, and so on. If we do this, we do it fresh.

[short_sells_poo]

I think they need to take a very serious and hard look at performance before it can go anywhere near VR (where rendering speed and stability are paramount). I'm sure it works for simple things and can handle GUIs fine, but the overhead seems huge currently.

[mlsarecmg]

there are many large scale apps built with it these days. it was initially made for complex use cases, to bring order into the scene graph, and of course to optimize raw rendering performance: https://docs.pmnd.rs/react-three-fiber/advanced/scaling-perf...

[pjmlp]

BabylonJS and PlayCanvas do it just right without jumping into React fashion.