[modeless]

There's no public model of the same size as the biggest GPT-3 yet, is there? I'd use GPT-3 to see what's possible, and then try to replicate the performance with the smaller public models. With the pace of AI development it's likely that GPT-3 will be matched by an open model in the not too distant future, but it's nice to be able to prototype with GPT-3 now to get a head start.

[robbedpeter]

Gpt-J is of comparable quality, within 7-10% of the performance of gpt-3 in almost all metrics. It's also much smaller and less expensive to run. The higher quality training data and better tweaks to the algorithm paid off - the license, restrictions, and cost of gpt-3 aren't necessarily valuable enough to justify not using gpt-j.

[stavros]

Hmm, that's very interesting! Do you know if there's a hosted service anywhere? I don't mind paying a few dollars a month for my small use case, but my usage can't justify the huge server it needs to run.

[harpersealtako]

NovelAI is already a hosted service you pay for. It is specifically used for fiction writing, though it's got a ridiculous amount of neat experimental features, from prefix tuning (a lightweight ad-hoc fine-tuning method which can make the AI write in a specific style based on a training dataset, you can train your own with a custom service they run too or just import one of the thousands other users already made) to keyword replacement for "memory" past the general context limit, to inline annotations ("author's note") which can steer the AI towards a particular direction, style, or theme.

That said if you just want to see how GPT-J-6B works there's a browser demo here: https://6b.eleuther.ai/

[stavros]

Excellent, thank you! Apparently GPT-J will run on a desktop as well (via Huggingface) but I think it needs slightly more RAM than my 16 GB.

[robbedpeter]

Huggingface.co is awesome!

You can also run it on colab. $10 a month buys you a lot of value with colab.

[stavros]

I'm not quite sure what Colab is, I'll have to look into it, thank you!

EDIT: Ah, looks like it's only available in a handful of countries, sadly.

[robbedpeter]

A good VPN would only cost another $10 or less each month, or you could set up a vps hosted in the US. A barebones Linode runs $5ish. Huggingface is definitely sufficient though, and if you're messing with local apis or tinkering, gpt-neo-125m can be run on cpu with under a gig of ram.