Hacker News new | ask | show | jobs
by slagfart 1172 days ago
Let's say I have about 100k novel-length texts I want to import into a model like this (I want to build a book recommendation system).

How should I approach it? I think this product can't deal with that much text yet?

1 comments

Also very interested in this. From what I read, you need to train some other model in the "transformer" family yourself, or you can use OpenAI API for fine-tuning their model. Not sure which would be better in practice, or if either would work, but AI people at my company say the fine-tuning with even just a bit of data can result in a lot better (or different) results for the given domain.