[burtonator]

The next major leap in LLMs (in the next year) is probably going to be the prompt context size. Right now we have 2k, 4k, 8k ... but OpenAI also has a 32k model that they're not really giving access to unfortunately.

The 8k model is nice but it's GPT4 so it's slow.

I think the thing that you're missing is that zero shot learning is VERY hard but anything > GPT3 is actually pretty good once you give it some real world examples.

I think prompt engineering is going to be here for a while just because, on a lot of task, examples are needed.

Doesn't mean it needs to be a herculean effort of course. Just that you need to come up with some concrete examples.

This is going to be ESPECIALLY true with Open Source LLMs that aren't anywhere near as sophisticated as GPT4.

In fact, I think there's a huge opportunity to use GPT4 to train the prompts of smaller models, come up with more examples, and help improve their precision/recall without massive prompt engineering efforts.

[throwawayForMe2]

>> The next major leap in LLMs (in the next year) is probably going to be the prompt context size. Right now we have 2k, 4k, 8k ... but OpenAI also has a 32k model that they're not really giving access to unfortunately.

Saw this article today about a different approach that opens up orders of magnitude larger contexts

https://hazyresearch.stanford.edu/blog/2023-03-07-hyena

[kiratp]

You can’t commercially use anything you train off OpenAI outputs.

[wanderingmind]

This is just ToS violation which will just result in loss of access to OpenAI. There is nothing they can do to stop you from commercially competing, given there is no copyright law precedence

[sebzim4500]

You can as long as the resulting model does not compete with OpenAI.

[kiratp]

It can be argued that if you build a model using their outputs such that you can then stop using their API, your model is effectively competing with their’s. Let’s just say that if you’re a startup or SMB, you do not want to be the one dragged to court to iron out whether this holds or not.

[rufius]

Can you elaborate?

[411111111111111]

They're probably talking about the TOS a user would've had to agree to when using their services. It's actually a lot more permissive then I expected

> Restrictions. You may not (i) use the Services in a way that infringes, misappropriates or violates any person’s rights; (ii) reverse assemble, reverse compile, decompile, translate or otherwise attempt to discover the source code or underlying components of models, algorithms, and systems of the Services (except to the extent such restrictions are contrary to applicable law); (iii) use output from the Services to develop models that compete with OpenAI;

[kiratp]

> use output from the Services to develop models that compete with OpenAI

It can be argued that if you build a model using their outputs such that you can then stop using their API, your model is effectively competing with their’s.

Let’s just say that if you’re a startup or SMB, you do not want to be the one dragged to court to iron out whether this holds or not.

[kiratp]

Their API TOS basically forbid it. Simple as that.

[MacsHeadroom]

Someone who acquires these outputs who has never consented to their ToS is not bound by their ToS.

[reissbaker]

Sure, but the ways of acquiring those outputs legally have vampiric licensing that bind you to those ToS, since the re-licenser is bound by the original ToS.

It's like distributing GPL code in a nonfree application. Even if you didn't "consent to [the original author's] ToS," you are still going to be bound to it via the redistributors license.

[zarzavat]

There’s no license. OpenAI is not an author of their models’ outputs, and they know it.

OpenAI can’t just start suing random people on the street without any legal basis. That’s how lawyers become not-lawyers.

There’s only a (somewhat dubiously enforceable) ToS contract between OpenAI and the user of OpenAI’s website. This is probably bullshit too - what legitimate interest does OpenAI have in a model output that doesn’t even belong to them, but it’s less obviously bullshit.

[Moissanite]

> Even if you didn't "consent to [the original author's] ToS," you are still going to be bound to it via the redistributors license.

In the context of the GPL, are there real examples of judgements which bind defendants to a license they never saw or knew anything about, because of the errant actions of an intermediary?

[ipaddr]

That means they can refuse you service. That's all. You can use or share the output

[MacsHeadroom]

No you are not, the outputs of an LLM are afforded no intellectual property rights protections.

The internet is increasingly covered with OpenAI's outputs and they do not have grounds for a claim to prevent people from using them.