|
|
|
|
|
|
by yousif_123123
3 hours ago
|
|
|
This was fun! And it told me I know 55k words which made me a little happy. I'm not sure exactly how you did this, but I think you asked an LLM to come up with the wrong options. Two things to consider: 1. While the LLM can go r good options, they won't be always hard to guess. I wonder if instead you can have the LLM generate very close words (or skip using an LLM entirely) and put those as the options.
2. If you will generate options with an LLM, make sure you are mindful of its inability to shuffle things around. The correct answer was overwhelmingly the first or second option in the list. You should ask the model to give the options in a uniform order (say from true meaning then decreasing amount of replayability), then manually shuffle them so that the probability of which option (A, B, C or D) is always 25%. |
|
|