Hacker News new | ask | show | jobs
by matsemann 924 days ago
How is the fine tuning actually performed? They have the data of drafts, and a prompt. But what does one do with it, more concretely?
1 comments

High level it's basically: 1. Generate a lot of text examples that look like this: https://gist.githubusercontent.com/davidhershey/f57d0b19563f...

2. The model is effectively trained to predict the next token based on the previous tokens in each of these examples, which has the side effect here of teaching it to make a draft pick based on the contents of a pack.

Nothing too fancy, just next word prediction more or less

Curious how different the performance would be if instead of a 'Hall of Famer' we tell the bot that it is decently-good, but will be deactivated if it can't achieve human-level performance...