[xiphias2]

I think it's an amazing experiment.

You can look at the PyTorch code base and understand a small local function instantly, but if I would have a task to have a deep understanding of either PyTorch with all kernel code and low level code, whole CUDA code base + LLVM compilation code or Tinygrad I would pick Tinygrad in an instant.

The code looks hard because what it is doing is hard, but all its layers can be debugged.

[johnnyfived]

Does doing hard work mean it shouldn't have any docstrings or comments? Cause that seems like an arbitrary way of making it "harder".

[xiphias2]

Docstrings and comments are amazing to be able to work with huge code bases with big teams where you don't want to understand the whole code base, because it's impossible.

If the goal is to not abstract anything away, and always look at the implementation itself to be able to change it / rewrite it, keep it tiny and it is able to beat PyTorch+CUDA+LLVM with 1000x smaller codebase, who am I to judge?