| This article makes no sense to me. If the premise was will Google AI Ethics research implode, then yes. Removing key memembers of the team has affected the team morale and created a toxic environment for the remaining researchers. However, the post seems to imply that stifling the Stochastic Parrots paper is somehow proof that Google AI stifles innovative research that is critical to AI progress. This is a quite a weak claim even before looking at logic of the claims that get to that point. Google AI, Google Brain, and Deepmind are all different groups at Google with different mandates and research goals. While what's happening in the Ethical AI team is troubling, it's rather a large and unfounded leap that it'll affect research productivity for the other teams. Digging deeper the article is confusing and sometimes plain wrong on its assessment of AI research. Broadly the deep learning and RL approach to AI has been critiqued for its lack of semantic and symbolic understanding. These are not Google specific and the articles examples of this are terrible. The first example of limitations of AlphaZero on Montezuma's revenge is a bad example. The author implies that RL failed because it didn't understand ladders. But later approaches still solved this using by using stochastic exploration strategies and not introducing conceptual knowledge to the model as the article implies is the key limitation. On the language modelling, its weird the article cites GPT3 as problematic given that GPT3 was developed at OpenAI's research and not Google. Also GPT3 is pretrained using next word prediction which only consider left context and is far more limited that BERT which considers bi-directional context and produces richer word level embeddings. That being said the Stochastic Parrots paper does specially critique BERT. But it's not a new critique. Emily Bender, the other major co-author, is a computational linguist who has always been critical of deep learning approaches to NLP. Bender along with Gary Marcus and many others have called for AI that considers symbolic and linguistic knowledge and have been critical of purely data-driven deep learning approaches. Stochastic Parrots is not new in its critique of large language models, it just provides newer evidence specific to the current state of language model research. So I'm not sure how any of this is a signal that Google AI is imploding. The broader trends in AI is not just throw more compute and make bigger models. It just happens that large models works well for OpenAI and Google for specific problems. Google also has one of the largest knowledge graphs and there is open line of research that combines symbolic knowledge from KG with deep learning methods. There is also active research both at Google and elsewhere that aims to add more make current deep learning approach more "intelligent" by using linguistic and symbolic knowledge. I'm confused as to how Google AI research is imploding. Google PR attempting to censor Stochastic Parrots (which was still published) because of bad PR optics has nothing to do with active research questions elsewhere at Google Brain, Deepmind, and Google AI. |