Hacker News new | ask | show | jobs
by dvt 78 days ago
> the thing JAX was truly meant for: a graphics renderer

I mean, just like ray-tracing, SDF (ray-marching) is neat, but basically everything useful is expensive or hard to do (collisions, meshes, texturing etc.). I mean mathy stuff is easier (rotations, unions/intersections, function composition, etc.) but 3D is usually used in either modeling software or video games, which care more about the former than they do the latter.
2 comments
BenoitP 78 days ago
I believe we could get there eventually. For example for collision there is work to make it differentiable (or use a local surrogate at the collision point): https://arxiv.org/abs/2207.00669

The robotics will need to connect vision with motors with haptics with 3D modelling. And to propagate gradient seamlessly. For calibrating torque with the the elastic deformation of the material for example. After all matter is not discreet at small scales (staying above the atomic scale)

All this will require all modules to be compatible with differentiability. It'll be expensive at first, but I'm sure some optimizations can get us close to the discreet case.

Also even for meshes there is a lot to gain with trying to go the continuous way:

https://www.cs.cmu.edu/~kmcrane/Projects/DDG/

link
Archit3ch 78 days ago
Games and simulations are typically stateful, I'm not sure the functional purity of JAX is a good fit.

Also, what's the story for JAX + WebGL when it comes to targeting hardware-accelerated ray tracing?

link
BenoitP 78 days ago
JAX is designed from the start to fit well with systolic arrays (TPUs, Nvidia's tensor cores, etc), which are extremely energy-efficient. WebGL won't be the tool that connects it on the web, but the generation after WebGPU will.
link
nightski 78 days ago
Maybe you mean mutation? State is inherently a part of functional purity. It's just handled explicitly instead of implicitly.
link