[dmakian]

> If I'm reading the author's writeup correctly, the prompt he's giving the agent at each pick contains only the names of the cards in its pool so far, and only gives the full text for the cards in the pack it's being passed. It doesn't look like context is being maintained between picks, presumably for context window size reasons.

Not quite -- there's a few ways the model learns the full card text:

* The models are trained on card trivia completions as well, where they're asked to complete the full text of the card as well as information about it (type, CMC, etc.)

* The models do still have to learn next token completion on the cards in packs, meaning they learn to predict the full text of the cards while making draft picks as well.

Net net, the bots learn the text of the new cards pretty comprehensively.

[apetresc]

Ooh I see! You do that with Mistral7B, I'm guessing? But not with the small GPT-3.5 trial you did?

[dmakian]

The two larger GPT-3.5 trials also got the card trivia examples, but like a bad scientist I don't have a great control group for those

[apetresc]

And also, since it seems you're the author, can you also clarify if your methodology allowed for the bot to track signals outside of the color-identity-count summary statistic you pass in the prompt? Something like allowing it to notice that a card has wheeled, or that a certain synergy piece was passed a few picks ago.

[dmakian]

Only the statistics you see in the prompt (which are clearly limited). I have a lot of ideas about how you could improve that context (most likely letting the AI record and track notes throughout a draft), but this one was relatively simple to implement. Definitely room for improvement!

[chc4]

Haha, I don't know anything about AI training but that's a really cute trick.