|
|
|
|
|
|
by mike_hearn
823 days ago
|
|
|
They do write CUDA code, oh boy do they ever. PyTorch is just a coordinator for CUDA or sometimes Metal kernels. New AI architectures and algorithms often end up needing a new or tuned kernel. Look at Flash Attention for an example of one of those that had a big impact. |
|
|