I don't think it was videos. Almost certainly it was replay files with a bunch of work to transform them into something that could be compared to the model's outputs. (Alphastar never 'sees' the game's interface, only a transformed version of information available via an API)
starcraft provides replay files that start with the initial game state and then every action in the game. Not user inputs, but the actions bound to them.