[buboard]

google seemed to make a genuine effort to make a model that is useful rather than record-breaking with bert. But i think it's wrong to consider it the "final" model upon which everything else will be built.

[bitL]

BERT is already outdated, but still useful as you need only 1 Titan RTX to retrain its BERT_large model via transfer learning.

[turnersr]

What methods make BERT outdated? Do you have pointers to other options?

[bitL]

e.g. XLNet:

https://arxiv.org/abs/1906.08237

[phreeza]

XLnet is Bert with a bunch of additional training tricks.

[bitL]

BERT is a Transformer with a bunch of additional training tricks. Transformer is self-attention with a bunch of additional training tricks...