| > My question was whether this supposed data was available to the whistleblower before he was fired or not. It is kind of important (see my first point). If you read the court case, you can see quite clearly that he never obtained any evidence, and simply hoped to go on a general fishing expedition. For him, it was like the AlphaChip authors performed a magic trick. Magic is impossible, and so something must be up. Even if he doesn't know how they did it, there must surely be fraud somewhere. Did it ever occur to him that he might just be wrong? The closest thing he had to "evidence" of his belief was his study where he says he tried AlphaChip and got worse results than SA. But even that study was flawed, and known to be flawed at the time. From "That Chip Has Sailed": "In 2022, it was reviewed by an independent committee at Google, which determined that “the claims and conclusions in the draft are not scientifically backed by the experiments” [33] and “as the [AlphaChip] results on their original datasets were independently reproduced, this brought the [Markov et al.] RL results into question” [33]. We provided the committee with one-line scripts that generated significantly better RL results than those reported in Markov et al., outperforming their “stronger” simulated annealing baseline. We still do not know how Markov and his collaborators produced the numbers in their paper." ([33] is the sworn statement from Jon Orwant, head of the independent resolution committee.) How large do you imagine the conspiracy is here? The method is secretly bad, but the TPU team uses it in production anyway? MediaTek built on it anyway? The TF-Agents team claimed to reproduce it, but they're actually lying? Jeff Dean put his reputation behind it but he's either fooled or lying? Please hold off on replying further on this thread until you have read OP's linked paper. I understand you want to defend your field against the encroachment of people you perceive to be bad actors, but I believe you have things backwards. |
This continues the holier-than-thou attitude, but also violates this site's rules.
Not only I have read the entire court transcript, I was following it with interest as it happened. My point still stands. It is sleazy, to say the least, to misquote the whistleblower. It is even more sleazy to base such quote on "something the defendant claimed that the plaintiff claimed in a civil lawsuit".
This is point #1 on this "critique" and already rings all alarm bells, which is what my comment was about.
Google also did many, many evils during the entire trial (e.g. through the entire SLAPP thing) that no wonder they did not want to expose to the public. But that's another story.
> How large do you imagine the conspiracy is here?
Not large. Google continuously publishes papers where the results cannot be reproduced, at least by the time the paper is published, and barely gets a slap on the wrist from Nature, which is supposed to enforce these things.
> The method is secretly bad, but the TPU team uses it in production anyway? MediaTek built on it anyway? The TF-Agents team claimed to reproduce it, but they're actually lying?
Neither Tensor nor Mediatek are examples of really high-stages design; Tensor is not know for beating any records and Mediatek specializes in value.
While my area is frontend rather than backend, I have a passing familiarity with automatic macro placement (due to the fact that the barriers between flow steps keep getting thinner every day). Almost every other chip house of large-enough size has a macro placer. Almost every PhD student has implemented one. It's an easy problem, and with minimal knowledge about your design style you can likely do a better job than the generic ones from major vendors. However, it doesn't matter if you have the best macro placement engine in the world, your results would still not be ground-breaking overall. You can in fact ALSO do a terrible job and still get a pretty decent result since the flow will compensate. This was already my informed opinion even before the Nature article, but for Google's method, which _requires_ another placement tool to refine its work, would be even more true. And Google's method even requires a GPU farm for something that is, while important, still a relatively short step in the entire placement flow (and for not very push-through performance designs like Google's or MediaTek's , which are like the bread and butter of the world, the entire flow would not have required GPUs before). It's no wonder than the whistleblower thought the entire point of this Nature article was for Google to sell more Google Cloud GPU time. They may have convinced Mediatek, but not the academic community.