[PartiallyTyped]

Sure enough LLMs are not going to win every kaggle competition. But... I am fairly certain that transformers may. Embed all categorical values, scale continuous features by embeddings, and run it through a graph neural network, with high probability it will beat nearly everything.

[janalsncm]

Transformers require a lot of data to converge. There’s a reason tree models are still king of kaggle even though transformers have been around for 5 years now.