[yacine_]

This is an ad. You'd be best served avoiding additional dependencies. At this point, you don't want to be trading off simplicity for ease. Even transformers + huggingface feels like too much bloat.

You can use this https://github.com/PygmalionAI/training-code

Or, you can use this; for QLoRA https://github.com/artidoro/qlora

The tools and mechanisms to get a model to do what you want is ever so changing, ever so quickly. Build and understand a notebook yourself, and reduce dependencies. You will need to switch them.

[Der_Einzige]

Huggingface + Transformers is and has been since at least 2018 the atlas holding up the rest of the NLP and pretty much all of the AI community.

Their unwavering commitment to open-source should be celebrated by all tech enthusiasts. Not sure why people poo-poo on them.

[chessgecko]

People who dislike things are just so much more vocal than people who like them. I've used Huggingface extensively, they are trying to do a lot, but its always been the most convenient/flexible for my finetuning use cases.

Thank you Huggingface!

[josh-sematic]

Yes, I'm a huge fan of Huggingface. There's a tendency to always distrust anybody that is a company that is trying to make money. But "makes some money with some of their product offerings" != "is incapable of producing valuable resources." It's always a balance, but I think Huggingface is doing well at both being a huge resource for the NLP community and having a viable business that allows them to keep being such a resource.

[ShamelessC]

The code is very enterprise oriented and reads more like Java than Python. Bootstrapping a VC backed company off of open source is a known strategy for achieving growth needed for future funding and acquisition.

At some point, all the nice things they offer for free or cheaper will go away or become expensive.

[yacine_]

Bloated does not mean bad. It means bloated. Which, for my purposes, makes it not the best choice.

[turnsout]

I'm fine with the Huggingface piece, but this joins the long list of blog posts that make it to the top of hn with the message "Easily fine tune an LLM! …by tying yourself to our proprietary platform"

[josh-sematic]

FWIW, this is not proprietary, it's all FOSS. And hopefully there's something interesting in there even if you don't use any of the tools mentioned.

[turnsout]

Fair enough—I was too lazy to click through to Sematic's site and see that it is indeed FOSS!

[winddude]

lol, qlora and pygmalion both wrap huggingface

[yacine_]

yes, unfortunately :(

did you know that the weight adapter happens on demand, on inference, in the LoRA forward pass function?

leaky, leaky, leaky

GGML will save us, surely

[ilaksh]

For the Pygmalion thing, what should we use for the LoRA parameters?