[nmfisher]

Either OpenAI or Microsoft demoed something similar to this some time in theist 12-18 months.

The fact that it’s not released (and also the fact that GPT-3 etc are still not publicly available) makes me suspect that these models are far too unstable for actual production use. It’s also why I’m getting a bit tired of these overhyped cherry-picked samples with seemingly nothing solid to ever back it up.

[dr_zoidberg]

> I’m getting a bit tired of these overhyped cherry-picked samples with seemingly nothing solid to ever back it up.

Most of the times a "fantastic GPT-3 result" is shown, you have to dig a bit and then you'll find out how it was primed[0] and how many different texts they had it generate. Then the one(s) carrying out the experiment go on and pick the most shocking writings. If you read all of the outputs (there are a few articles around that show you 5 or 6 different outputs) you'll see the variations that it took duing those. I understand that 5 or 6 is actually small, to get shocking results they usually go into de dozens of tries.

[0] usually the priming phrases are given, but depending on how much of a snake-oil-salesman the person writing/giving a talk is, they may even hide this part

[danielheath]

I use tabnine and it does a pretty nice job by using AI to re-score the outputs of (language-server + similar-words-in-codebase).

[visarga]

They are too expensive to run - hundreds of GB of GPU memory - so they can't be deployed for the public at large yet, kind of like the SGI workstations from 20 years ago. You can do that and more for cheap today, but not then.

I think we can get models about 1/100th the size for general use. That's also the main reason Google is developing TPUs.

[nmfisher]

> They are too expensive to run - hundreds of GB of GPU memory - so they can't be deployed for the public at large yet, kind of like the SGI workstations from 20 years ago. You can do that and more for cheap today, but not then.

I don't buy this. OpenAI literally released pricing for GPT-3, so either they grossly miscalculated their cost base (unlikely) or there's some scaling/instability/resourcing issue preventing them from doing so (much more likely).

I think it's telling that they spent the last 6 months on yet another flashy demo (DALL-E) rather than actually productionizing GPT-3. It just feels like constant smoke and mirrors.