[robmaister]

I'm not familiar with the specifics within gfx-hal, but it's most likely taking in SPIR-V and translating as needed.

AFAIK DX12 still requires DXIL so it might be saving both SPIR-V and DXIL, or taking in HLSL and compiling/translating as needed.

Microsoft has generally gotten a lot friendlier in the last decade, they have tools to compile HLSL down to SPIR-V, if you want to support everything including DX12 you'll probably be writing HLSL. Otherwise you can pick any language that compiles down to SPIR-V.

https://github.com/Microsoft/ShaderConductor

https://github.com/KhronosGroup/SPIRV-Cross

[chubs]

So from what I can find, it seems gfx-hal works with GLSL, compiles it to SPIRV, and then uses that (somehow). I'm just not sure how it uses the SPIRV for each backend.

[robmaister]

The first link I included has a wonderful diagram of this (but from the perspective of HLSL instead of GLSL), the gist of it is:

GLSL gets compiled to SPIR-V ahead of time by a compiler of your choice (probably glslangValidator). You can take that binary blob and feed it into Vulkan (vkCreateShaderModule) or OpenGL (glShaderBinary as long as you have the right extension).

For everything else, Khronos has a tool called SPIRV-Cross (second link), which reads the SPIR-V binary data and emits a text file/string in ESSL for OpenGL ES, MSL for Metal, or HLSL for DirectX <=11. Those all go through the "normal" paths for loading shader code in their own APIs.

[chubs]

Thanks so much for explaining how that works :)

Is it just me or does decompiling the binary SPIR-V to ESL/HLSL source code then recompiling sound like a recipe for massive inefficiency? Or in practice does it work out pretty nicely?

[robmaister]

Yes, it's absolutely inefficient but for those platforms it's the only way to execute your own code on the GPU. Metal has a bitcode format that I know nothing about and I believe older DirectX had some intermediate format that was binary. Both are proprietary and only documented via reverse-engineering, so they're not great targets.

Most of the extra cost of feeding in SPIR-V could also be offset if you generate the text shader code at compile/packaging time so that those builds don't have the original SPIR-V in them.

[kvark]

It's not absolutely impossible. DXBC was well reverse engineered, and DXIL could also be a direct target for translation. It's up to future work, which for us also includes writing an in house shader converter as a substitute to SPIRV-Cross.

This isn't a big concern at the moment though, since most of the shader loading/compiling time is spent by the driver receiving the result, not us translating it.

Generating a text shader at packaging time is certainly an option to explore, especially for self-contained users like WebRender. The trouble here is that we adjust the shader code based on the pipeline and pipeline layout, so we can only really start translation at the run/init time, unless we start pre-packaging a set of "popular" configurations (which is feasible for WebRender).

[kvark]

The API works off SPIRV, it doesn't know anything about GLSL. The latter is only used by the examples for convenience.

[rossy]

Take another look at SPIRV-Cross. It's a C++ library that can convert SPIR-V to HLSL, MSL or legacy GLSL/ESSL. I'm not too familiar with Rust or gfx, but it seems like the dx11, dx12, metal and gl backends reference the spirv_cross crate, so that's probably what it's using.

To try to answer your original question, I've never used gfx, but we use SPIRV-Cross in mpv to translate video processing shaders written in GLSL to HLSL for Direct3D 11, and it works quite well. Modern GLSL and HLSL seem to be pretty similar, apart from using different coordinate systems. Most HLSL concepts directly map to a GLSL concept (eg. GLSL UBOs become HLSL cbuffers, etc.) Shader translation with SPIRV-Cross definitely isn't the slowest part of our shader compilation pipeline, either. D3DCompile is much slower.