[ilaksh]

I have a couple of ideas.

1. Have it do a thinking/brainstorming phase first to try to work out what the potential categories are.

2. Then ask it to scan over each word and think about what categories it could go in, in order of likelihood.

3. Ask it to do the final answer.

Format the training set in that way, as if it got everything right at each step (since you only have the right answers).

It sounds like you had 7 * 30 = around 200 examples. Maybe you can feed a batch of ten at a time and explain the game and try to get GPT-4 to generate more examples. You will have to see if they make sense.

I assume that by increasing the size of the dataset by a factor of ten, and having the LLM think through the problem using multiple steps, you will get significantly better results.

[SamBam]

I spent about 30 minutes with GPT4 and tried lots of variations of pre-processing. I had it first list large numbers of possible categories, then try to consider one category with four words, then double-check that category and look at the remaining words, then go ahead with the next....

No matter how I instructed it to think, it frequently could not work out the very first category.