[neomantra]

This is a great read, thank you for sharing! I learned a lot from reading it and I'm pretty experienced in this space! I enjoyed checking out your other work too.

OPs article heavily covers Vertex Amplification. I didn't realize VisionOS used Vertex Amplification for this, but it makes sense since the hardware supports it.

For those interested in Mesh Shading (a few good videos released this week [1][2][3]), Vertex Amplification is a key tech for Mesh Shading, where one writes Object/Mesh functions (names in Metal API, called Task/Mesh/Amplification shaders in other APIs). Introduced by Nvidia in 2018 and only really available en masse for the last couple years, Vertex Amplification was the first time a GPU could create vertices (or "destroy" them by not emitting anything), versus the fixed mesh inputs. It's so cool and powerful and a different way of thinking about the pipeline.

This article shows the same concept, but in Vertex Shaders for multiple render targets. While you might not make a Vision app, it could be worthwhile read to further understand this architecture. I've spent a few months in Metal Mesh Shading and hadn't realized this application of it at all.

[1] https://news.ycombinator.com/item?id=41839190

[2] https://www.youtube.com/watch?v=3EMdMD1PsgY

[3] https://www.youtube.com/watch?v=EtX7WnFhxtQ (good explanation and demo)

[dagmx]

Nit: amplification shaders are not the same as mesh shaders (though in many cases they can be seen as abstractions over one)

Your description is of a mesh shaders, but an amplification shader is able to basically reuse vertex data from a vertex shader pass without use of a mesh shader.

[cubefox]

Nit: Mesh shading, or the mesh shader pipeline, has two unique parts: optional task/amplification shaders, and mandatory mesh shaders.

[neomantra]

Thanks both for pulling on this point. In drafting, I was both torn and fuzzy about the nomenclature. Also trying to be distinct about hardware features (dynamic vertex data?) versus language/API constructs. But you are right, it's important to be clear because they are different and hardware might not support both.

I was trying to see if the other graphics APIs (Vulkan, DirectX) had this Vertex Amplification in Vertex Shader feature, but it doesn't seem so? Maybe it was easier for Apple to inject the concept into Metal (advantage of controlling of the whole stack).

[cubefox]

I'm not sure this is relevant but apparently Apple introduced proper mesh shading only with the M3 generation chips:

> The [M3] GPU is faster and more efficient, and introduces a new technology called Dynamic Caching, while bringing new rendering features like hardware-accelerated ray tracing and mesh shading to Mac for the first time.

https://www.apple.com/ne/newsroom/2023/10/apple-unveils-m3-m...

That means the M2 (which is inside the Apple Vision) must have at most some sort of partial mesh shading support.

[dagmx]

AFAIK Apple support fallback to compute shader based implementations of both raytracing and mesh shading on older GPUs. It’ll be slower, but for something like amplification shaders the win would be worth it

[neomantra]

I actually researched this a bit right when it came out.... I can't find my source now, but Vision Pro has its own SOC with an M2 CPU with a "2.5" GPU (between M2 and M3 families, like M2 with some goodies). Oddly, Vision isn't discussed in the handy Metal Feature Set Table [1].

The GPU Family Apple7 was the first with Mesh Shader support, but as you noted it might be software fallbacks down there.

I did just check with my Vision Pro and I get MTLGPUFamily [2] up to apple8, which does correspond to other M2 devices. I probed a max amplification count of 8.

[1] https://developer.apple.com/metal/Metal-Feature-Set-Tables.p...

[2] https://developer.apple.com/documentation/metal/mtlgpufamily