[Endy]

Given the information that this AI now has and the ability it's displayed, should WotC/Hasbro consider actively banning players from using AI to build standard-format Constructed decks? After all, if the author actually is correct and draftbot here picks better than humans, wouldn't it stand to reason that it could build optimal decks if given a "draft" which included multiple iterations of the program working in tandem over an entire set?

[phamilton]

I'm not super well versed in MtG (I've played, but never competitively), but a similar thing is common in Dominion. Any discussion about strategy in a particular game is usually settled by running a simulator pitting two strategies against each other a few thousand times. Eventually, these simulators have made their way into play testing potential new cards.

At the end of the day, these are games sufficiently complex that no single strategy will be optimal. Dominion has something on the order of 10^15 possible game configurations. Simulations are just a tool available to prepare and test potential strategies. I suspect MtG would land in a similar space if an "optimal deck builder" surfaced. You still have to play the game and beat your opponent.

[c256]

Not at all.

Constructed players already (for a very long time) use the collective data of other players — this has lots of names, it is commonly called “net-decking”. It turns out, that the best players and the best deck builders have a lot of overlap, but not completely so, and it’s common for multiple high-ranking players to play the same decks. A key skill to winning tournaments is to figure out what decks are going to be played by the other top players, and how to give yourself the best chance against them.

In other words, MtG is a game where deck-construction and deck-playing are both important and distinct skills, and high-level tournaments generally assume a high level of both, so a helper bot for the former is unlikely to give an advantage (because it’s compared to “the best efforts of a lot of people over a lot of time”).

[YeGoblynQueenne]

The author evaluted his model's "accuracy" by looking at how good it was at reproducing the decks in its training set card-for-card.

That is no way to evaluate the quality of an M:tG deck. For instance, it can never tell us anything about "sleeper" decks, or about the value to an existing deck of new cards that are added to a format as sets rotate and so on.

All that the accuracy metric used by the author can do is tell us how good the model is at representing the past. I am of the firm belief that WoTC will be laughing in their tea cups in the tought of banning something as pointless as this. In M:tG the past is about as valuable as a hat made of ice in the tropics.

Edit: For some added context. The way M:tG metagames work is that at the start of a season, there are some "decks to beat" that are usually the most obvious ones in the format. As the format progresses, players often find strategies to beat the decks to beat, initially known as "rogue" or sleepers. These can't be predicted by representing the current decks to beat. Some level of understanding of the game and what's important in a format in terms of strategy etc, is required.

Famous example. "The solution" by Zwi Mowshowitz [1] that dominated the 1st Pro Tour–Tokyo 2001 Invasion Block Constructed. Mowshowitz noticed that the dominant aggro decks' clocks (sources of damage) were predominantly red, so he stuffed a deck with anti-red cards, shutting down the dominant aggro decks.

That requires way, way more than modelling the current metagame at any point in time.

_____________________

[1] http://www.starcitygames.com/magic/misc/22483_Innovations_Th...

[c256]

I think you’re being too harsh here. The author didn’t come close to claiming that MtG drafting was solved; instead the claim is that they were able to train a fairly straightforward model to perform relatively well — within the metric of “as hand-judged by a proficient player” — to draft cards.

There is a vast amount of interesting space (both as AImresearch and as a fun toy) between “fully groks the current and predicatable near-future meta on a professional level” and “drafts playable decks”. As I read it, the OP is saying “Hey, I built something surprisingly simple that can draft playable decks. When I compare it to an available corpus of draft data from random human players, I think it does as well or better.” On the other hand, I don’t see anything like “I built AlphaGo for Magic drafts.”