[smodad]

I just realized that Justine was the person responsible for the massive reduction in the memory footprint of the Llama models back in March.[1] Super impressive! These are my favorite kinds of blog posts.

[1] https://github.com/ggerganov/llama.cpp/pull/613

[gajnadsgjoas]

You wanted to say the one was banned by the author because of all the drama that followed

[jimsimmons]

What drama. Ooc

[pcj-github]

https://github.com/github-drama/github-drama/pull/46

[skeaker]

I don't understand this. What's the contention that's actually getting people upset here?

[nl]

Having read through a bunch of outraged comments the issue seems to have been:

  * It changed the file format
  * For some people it was slower that the original version particularly on low end computers

One person was particularly outraged and reverted the whole change.

However, the current version does use a similar approach to that which was proposed: https://github.com/ggerganov/llama.cpp/commit/f963b63afa0e05...

[nicman23]

Lol why oss people are so antisocial - I include myself in the subset

[m00x]

It's just using mmap, nothing too impressive. It's a nice contribution nonetheless.

[dundarious]

There was more. You can't just splat giant C structs with pointers into shared memory/a file, and expect another process to just mmap and be able to recreate valid state again. At the very least the pointers are going to be all wrong. There was necessary work to adjust the file format. Not rocket science, but not just turning while(fread()) into open();mmap().

Also, there were insights into how to minimize which models needed adjustment. The ideas and code were worked on by at least 2 people, and I'm an outsider on that project, but I didn't see anything untoward like "stealing credit". The magic change wasn't a perfect move, but is the kind of thing I do locally when I don't know the project/binary format well yet, so not exactly the megalomaniacal move it was painted as. Better that only the version number changed, but she's independent and doing good work, so you'd kind of hope she has a self-promotion streak! Changing the magic would be on the very very low end of letting that side go a bit too far, assuming that was the impetus.

[CaptainNegative]

Why is it Justine posts and seemingly only Justine posts that always get this type of comments? Do people regularly comment on the authors of other content, for better or for worse, and I miss it?

[Conscat]

Faster Than Lime, too.