Hacker News new | ask | show | jobs
by mk_stjames 1020 days ago
I don't see how the loading works for the end user's custom dataset. In fact, I find the layers of abstraction you have between getting the finetuning dataset and the actual training very opaque. I can't even tell where the dataset is coming from, it doesn't appear to be an example local to this repository.

I think a lot of people what something like... "drop .txt files of example data to train on in this /folder/ and run python finetune.py /folder/
3 comments
rsaha7 1018 days ago
You are right in that the loading is right now on huggingface’s dataset. The feedback about it being opaque has merit, and we are working on giving users more control and visibility into the dataset loading. To your point, adding instructions about how one can load their own dataset and do fine-tuning can assist researchers better in leveraging these models. That being said, the README under each model folder has all the info one needs to get started.

More than happy to have you contribute to the repo. There’s a lot of exciting work to be done.

link
Charon77 1018 days ago
It's hardcoded, hidden within the code there are calls to huggingface's datasets.load_dataset, but you don't get to specify on your own..
link
rsaha7 1018 days ago
Great observation. We are working on making this part very explicit. The goal was to let researchers get up to speed with the codebase to begin with, and then they would understand what needs to change to make these models work on custom datasets.

That being said, we are working on adding instructions to specify dataset and also the prompt that users want to use.

link
vorticalbox 1020 days ago
This is actually what I was hoping for. For Web UI that you can load a model then load some data and hit train.

You can do this in the stable defusion UI to fine tune models with your own dataset

link
rsaha7 1018 days ago
Feedback taken. We are working on making it more explicit for users to mention the task and dataset they want to train models on. Additionally, we will introduce a flag to let people mention the prompt they want to use for finetuning these models.
link
mikeravkine 1020 days ago
OobaBooga supports this kind of load-and-go LORA: https://github.com/oobabooga/text-generation-webui
link
rsaha7 1018 days ago
Looked at the project. Great initiative.
link