[Const-me]

For implementing stuff from scratch, if you use Windows you could try my C# based library for that: https://github.com/Const-me/Cgml/

It’s vendor agnostic, so HLSL instead of CUDA or Triton. Here’s the compute shaders implementing inference of Mistral-7B model: https://github.com/Const-me/Cgml/tree/master/Mistral/Mistral...