[bavell]

Any tips or guides you followed on training your custom model? I've done a few LoRAs and TI but haven't gotten to my own models yet. Your results look great and I'd love a little insight into how you arrived there and what methods/tools you used.

[kouteiheika]

I'm not an expert at this and there are probably better ways to do this/might not work for you/your mileage may vary, so please take this with a huge grain of salt, but roughly this worked for me:

1. Start with a good base model(s) from which to train from.

2. Have a lot of diverse images.

3. Ideally train for only one epoch. (Having a lot of images helps here.)

4. If you get bad results lower the learning rate and try again.

5. After training try to mix your finetuned model with the original one, in steps of 10%, generate X/Y plot of it, pick the best result.

6. Repeat this process as long as you're getting an improvement.

For training I mostly used scripts from here: https://github.com/bmaltais/kohya_ss

The main problem here is that essentially during inference you're using a bag of tricks to make the output better (e.g. good negative embeddings), but when training you don't. (And I'm not entirely sure how you'd actually integrate those into the training process; might be possible, but I didn't want to spend too much time on it.) So your fine tuning as-is might improve the output of the model when no tricks are used, but it can also regress it when the tricks are used. Which I why I did the "mix and pick the best one" step.

But, again, I'm not an expert at this and just did this for fun. Ultimately there might be better ways to do it.

[bavell]

Great tips, thank you! It feels like I'm right behind you in terms of where I'm at so your input is very much appreciated.

3. Train for only 1 epoch - interesting, any known rationale here?

5. I just read somewhere else that someone got good results from mixing their custom model with the original (60/40 in their case) - good to hear some more anecdotes that this is pretty effective. Especially the further training after merging, sounds promising!

I've also been using kohya_ss for training LoRAs so great to hear it works for you for models as well. On your point about the inference tricks, definitely noted but I did notice that you can feed some params (# of samples, negative embeddings, etc) to the sample images generated during training (check the textarea placeholder text). Still not going to have all usual the tricks but it'll get you a little closer.

[zirgs]

Make sure that you have enough vram. I can train loras with 8 gb easily, but when I tried to train a model - it gives me an oom error.

[bavell]

Hopefully my 12GB is enough!