|
|
|
|
|
by ricksunny
40 days ago
|
|
Vaswani, A., et al. (2017) Attention Is All You Need. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, 4-9 December 2017, 6000-6010. Generally understood to be an output of Googlers. |
|
Basic research would be something like optimal control theory, which came well before the transformer design.
I'm not trying to be evasive; I can see how my distinction could be seen as conveniently just outside industry's purview. Put it this way: I think companies, particularly small ones, are incentivized to pursue well-known methods/materials. Innovation modulates and optimizes.