[cantaloa]

These kinds of articles really need to answer "compared to what?" at which point they immediately cash out into nothing.

So, 185,000 gallons of water were used to train GPT-3 which is apparently enough water to grow just 100 lbs of beef or 210 gallons of milk.

(420,000,000 lbs of beef are discarded in the US every year just due to discoloration)

That seems like a trivially good deal.

[ALittleLight]

I saw similar faux outrage over carbon emissions from training large models. For example, this paper[1] involves hand wringing over the fact that training an original BERT model could have similar emissions to a trans-Atlantic flight. That despite the fact that thousands of such flights occur daily and BERT is (was?) used in a majority of all Google searches.

It actually seems like they mean a single flight as in, a single person's share of CO2 emissions from a flight, not like, the whole flight itself. Their reference for the emissions is this paper[2] which says BERT took 80 hours on 64 v100's and estimates that as 1.4k pounds of CO2 compared to Googling which suggests a single person's share of a flight from the US to the UK is around 1k kilograms of emissions (and about 10-20 dollars of carbon emissions offsets).

Some people just want to find things to complain about.

1 - https://dl.acm.org/doi/10.1145/3442188.3445922

2 - https://aclanthology.org/P19-1355.pdf