[marmaduke]

> Unlike most prior efforts, we release not only model weights and inference code, but also the reproducible training data, complete data processing pipeline, rigorous experimental ablation results, and detailed training protocols for open scientific research.

Regardless of the specific performance of this model versus another model, I think it’s good to keep in mind that everyone benefits from this kind of work

[WiSaGaN]

This is probably the closest you can get as an "open source" model.

[perching_aix]

Is this not what's referred to as an open-data (& open-source) solution?

[lordswork]

When it comes to how "open" LLMs are, there are basically three categories:

- Open Source: The architecture of the model is available, so you can bring your own data and compute to train a similar model.

- Open Weights: The trained model itself is available for you to use. You can download it and run it on your own hardware without needing to train anything from scratch.

- Open Data: You get access to the data that was used to train the model. With this, you can completely reproduce the training process, which is super helpful if you want to fine-tune the model or see exactly how it was trained.

There are some other nuances around hyperparams, training methodology, etc. but that's basically the 3 main categories.

[bick_nyers]

I would probably refer to category 1 as "Open Architecture". I wouldn't want to give anyone the false impression that category 1 is comparable in the slightest to Open Weights, which is vastly more useful.

[lordswork]

I like that better too, but open source is often used synonymously with open architecture, so it's good to be aware of.

[perching_aix]

I see, thanks! I didn't know being open source referred to so little when it came to ML.

[szundi]

The data is a kind of “source” in the original meaning when source code was named.

I think open-weight is somewhere between open source and binary.

Reason: cannot be reproduced or practically modified without the source data collection.

[chvid]

Open weight is binary.

The difference between metas llama and open ai is akin to the difference between ms excel installed on your machine and google sheets running in the cloud.

[fjdjshsh]

Hard disagree and it sounds like you're extrapolating from the traditional software area without considering the nuances.

From the point of view of academia, free transformative products that build on other stuff, transparency, possibility of building even better models in the future and a big etc, there's NO difference between ms excel and Google sheets.

OTOH, the llama models have allowed all of the above and has helped us immensely in both developing new things and being able to understand these new generation of LLMs...all of which would be impossible with openai.

Open weight means you get both the details of the architecture and a way to freely iterate to build new things.

[rfoo]

People have different definition of what "freely iterate to build new things" means. For me, having a binary only does not prevent me from doing so.

For example, Minecraft was never distributed with source code, it was binary-only from day one. But the modding community would hard disagree with you if you say there was no way to "freely iterate to build new things", probably in GenZ term, "skill issue" :p

[perching_aix]

> From the point of view of academia

... but they did not specify their pov?

[mistrial9]

agree but be sure to check for microphones in the flowerpot