[allencheng]

I agree prompts make a huge difference in how well GPT-3 performs. The relevant questions are:

1) how easy is it to replicate best practices? If it's simple reformatting and everyone knows how to do it, then the playing field levels off quickly.

2) what does the improvement curve look like - how far can you push performance through better prompts before you get diminishing marginal returns?

It's too early to tell and we'll need more samples to figure this out.

[mlb_hn]

I don't think it's going to be about a single prompt; reverse engineering multiple prompts interacting with themselves is hard. There's a lot of cool things to be done with:

(a) creating a pipeline of prompts that combine outputs of previous prompts into new prompts in a predefined manner

and (b) designing prompts to generate other prompts

[goldenkey]

With the right type of online learning and possibly some of the weights frozen, GPT-3 could gain an unlimited memory instead of the fixed 2048 token memory.