[ansk]

I can see how LLMs may make compiler development seem futile when it comes to optimizing the local graph structure (ie kernel-sized chunks of the computation).

But optimizing the global structure -- things like defining the boundaries between kernels, the lifetimes of allocations (or when/how to reuse buffers), and scheduling/parallelizing operations -- seems inextricably tied to the type of traditional program analysis used by compilers.

Even tiling/layout decisions, which you mentioned can be optimized by hand or by an LLM in the context of a single kernel, must be optimized in the broader context of the computation to minimize copies/relayouts between kernels.

I think part of the problem is that massive models make the compiler problem much less interesting, since global program optimization reduces to local kernel-level optimization.