| Sigh. > The Economist [...] said that GPT-2’s answers were “unedited”, when in reality each answer that was published was selected from five options > [Erik Bryjngjolffson] tweeted that the interview was “impressive” and that “the answers are more coherent than those of many humans.” In fact the apparent coherence of the interview stemmed from (a) the enormous corpus of human writing that the system drew from and (b) the filtering for coherence that was done by the human journalist. If your success rate is ≥20%, the coherence is coming from the model, not the selection process. This is just basic statistics. > OpenAI created a pair of neural networks that allowed a robot to learn to manipulate a custom-built Rubik's cube Jeez, I've already corrected you here... well, why not have to do it again? > publicized it with a somewhat misleading video and blog that led many to think that the system had learned the cognitive aspects of cube-solving The side not stated: OpenAI said explicitly in the blog that they used an unlearned algorithm for this, and sent a correction to a publisher that got this wrong. > the cube was instrumented with Bluetooth sensors During training, but they ended up with a fully vision-based system. > even in the best case only 20% of fully-scrambled cubes were solved No, 60% of fully scrambled cubes were solved. 20% of maximally difficult scrambles were solved. > one report claimed that “A neural net solves the three-body problem 100 million times faster” [...] but the network did no solving in the classical sense, it did approximation All solvers for this problem are approximators, and vice-versa. The article you complain about states the accuracy (“error of just 10^(-5)”) in the body of text. > and it approximated only a highly simplified two degree-of-freedom problem As reported: “Breen and co first simplify the problem by limiting it to those involving three equal-mass particles in a plane, each with zero velocity to start with.” > MIT AI lab famously assigned Gerald Sussman the problem of solving vision in a summer [https://dspace.mit.edu/handle/1721.1/6125] I... sigh “The original document outlined a plan to do some kind of basic foreground/background segmentation, followed by a subgoal of analysing scenes with simple non-overlapping objects, with distinct uniform colour and texture and homogeneous backgrounds. A further subgoal was to extend the system to more complex objects. So it would seem that Computer Vision was never a summer project for a single student, nor did it aim to make a complete working vision system.” http://www.lyndonhill.com/opinion-cvlegends.html > Geoff Hinton [said] that the company (again The Guardian’s paraphrase), “is on the brink of developing algorithms with the capacity for logic, natural conversation and even flirtation.” Four years later, we are still a long way from machines that can hold natural conversations absent human intervention ‘Four years later’ to natural conversation is not a reasonable point of criticism when the only timeline given was ‘within a decade’ for a specified subset of the problem. > [In 2016 Hinton said] “We should stop training radiologists now. It’s just completely obvious that within five years, deep learning is going to do better than radiologists.” [...] but thus far no actual radiologists have been replaced So Hinton actually said “People should stop training radiologists now. It’s just completely obvious that within five years, deep learning is going to do better than radiologists, because it's going to be able to get a lot more experience. It might be 10 years, but we've got plenty of radiologists already.” 2019 is not 2026. “thus far no actual radiologists have been replaced” is thus not a counterargument. > Andrew Ng, another well-known figure in deep learning, wrote that “If a typical person can do a mental task with less than one second of thought, we can probably automate it using AI either now or in the near future.” [...] Ng’s claim thus far has proven incorrect. I agree. This quote captures the wrong nuance of the issue. Well, finally finding one point by Gary Marcus that isn't misleading, I think I'm going to call this a day. |
https://www.reddit.com/r/MachineLearning/comments/e453kl/d_a...