[behnamoh]

I find these efforts impressive, but what is the value proposition here? (I'm not just talking about this fork, but also Karapathy's llama2.c as well).

[atairov]

Personally for me the value was to implement a complex logic from a scientific paper in a pure Python. It helps to understand the essence of a cutting edge AI technology. And it's quite fascinating that it would take about 500 lines of core part code to implement inference for such a complex solution.

[atairov]

Regarding the original llama2.c as I believe the value proposition is to have simple implementation that can execute the inference locally on wide variety of platforms. What if we can execute fine-tuned Llama7B on our phones?

[brucethemoose2]

> What if we can execute fine-tuned Llama7B on our phones?

7B and 13B are already quite performant with mlc-llm (which uses an Apache TVM Vulkan/Metal backend). Llama.cpp has the potential to perform well too.

These "single file" implementations are not meant to be optimized or feature rich, I dont think.

[brucethemoose2]

Its educational. It shows a how llama works in a clear, concise, testable way.

[westurner]

Writing one's own and/or porting every line of code has great value