[paulsutter]

Of general interest, Jeff Dean’s comment on why the Gebru paper didn’t meet Google’s publication guidelines:

“Highlighting risks without pointing out methods for researchers and developers to understand and mitigate those risks misses the mark on helping with these problems”

from https://docs.google.com/document/d/1f2kYWDXwhzYnq8ebVtuk9CqQ...

[dominotw]

Isn't that asking too much. How would a researcher also know how mitigate the problems they identified.

[refenestrator]

The problems they identified have been known for years and there are lots of papers exploring how to mitigate.

The whole paper was a nothing burger wrapped in social justice language with asides about how global warming is Actually Racism because of disparate impact (interesting, not an ML topic).

If the problems aren't novel and you're proposing zero solutions, it shouldn't be a paper.

[jl2718]

I wonder what would drive someone to do this. Anger? Loneliness? Work pressure? Imposter syndrome? Her prior work seemed more observational than theoretical, and the she got thrown into a top-level leadership position in a theoretical research organization with almost zero experience. Is there a single person that would have defended decision if that person was a straight able christian white or north asian male? (Note: please do not respond by speaking for others that do not share your own views.)

I wonder when the “performative wokeness” bubble will burst.

[Traster]

I don't know where you get these ridiculous ideas about imposter syndrome. Take one look at her career. She's written signal processing algorithms at Apple, won awards at conferences, got a PhD and worked for Microsoft in their AI ethics lab. She was more than qualified for her position and the research paper she was publishing passed peer review.

You're right, this wouldn't have happened if she were a straight white male - if she were a straight white male, there wouldn't be comments like yours making asinine assumptions about how good she was at her job.

[refenestrator]

An impressive career makes this paper's quality and the subsequent meltdown over it more puzzling, not less.

[jl2718]

Non-sequitur. I invite you to disabuse me of any of my exact statements.

[skinkestek]

> You're right, this wouldn't have happened if she were a straight white male

If anything, straight white males are trashed harder around here.

That's the only problem I've ever had with having a woman on the team: the fear of saying something that offends someone, her or other.

[throwaway19937]

This is #4 (Pollution of Agency) and #8 (Anomalousness) from Joanna Russ's How to Suppress Women's Writing (https://en.wikipedia.org/wiki/How_to_Suppress_Women%27s_Writ...).

Poorly written papers are a regular occurrence in any field - that doesn't justify attributing bad faith or character issues to Dr. Gebru.

[ausbah]

survey papers, reproducibility papers, and the like are still valid contributions to science

[mlthoughts2018]

No, it is not too much to ask in the specific case of Gebru’s bad paper. Several of the arguments are specious, like comparing the total energy consumption for training GPT with car trips, or demanding that NLP researchers have to keep up with rapidly changing activist “woke” vocabulary and ensure their models are respecting it.

These are ridiculous claims, and it’s fair to respond to them by saying, “well, what exactly do you imagine a solution or mitigation looks like?”

Essentially, by the nature of how specious Gebru’s stated problems are, they demand clarity over what an “ethical solution” even is, conceptually, and why everyone would have to agree.

For example, you could discuss economies of scale or train-once-finetune-everywhere approaches with GPT that reduce total energy needs. Or you could discuss how researchers can register the corpus they use and the snapshot of time it was grabbed, with an open understanding that as long as the methods and data are reproducible, there is no research ethical issue with studying that corpus, no matter how much bias or lack of woke vocab a given person believes it has. (And also, nobody is required to just accept activist language as important or valid.)

Gebru did none of this. The article could literally be summed up by Gebru saying, “I think <supposedly shocking evidence> is bad, therefore its connection to something in ML is bad.”

E.g. “I think, subjectively, that the raw energy use to train GPT is bad. Here are some shocking comparisons. Therefore GPT is bad.”

It’s incredibly unrigorous and juvenile. Dean’s comments that it needs to clearly state mitigations is actually a super generous, polite way of saying the paper is just subjective amateur hour.

[jgalt212]

Isn't that what the "directions for future research" section is for?

[krallistic]

Sure if no methods/work exists. In this case there already has been some work done, to mitigate these issues. So either mention them in related work and/or highlight why theses methods arent enough.

[sp332]

Of course. Especially to expect all that to be in each individual published paper.

[ausbah]

wasn't the real issue that the standards they cited were quoted later in the process than normal