[heresie-dabord]

The objections seem clear: tight-coupling of prompts to models, and model neutrality in the TOU.

From https://github.com/mozilla/standards-positions/issues/1213 :

"A personal example: I created a system prompt for creating announcements for a home automation system. The Gemini model I was using initially responded in a very US-American way, which didn't fit the British voice of my speaker. I told the model, via the system prompt, that the output was being spoken in a British voice, but the result was a bad US-American impersonation of British ("a'waight guv'nor apples and pears" etc etc), so I had to iterate further to 'tone it down' and speak actual British.

In this process, the system prompt becomes tailored to the model. Other models will have different quirks. Things added to the system prompt for one model may be an overcorrection for another."

[stronglikedan]

> but the result was a bad US-American impersonation of British ("a'waight guv'nor apples and pears" etc etc)

sounds like adversarial mode mocking

[stuaxo]

I've got this result pretty consistently from different models.

[jauntywundrkind]

This is a joke of an objection.

This API is a neutral party and can be iterated on. These objections are particular objections, to an implementation detail. One that can be swapped out, iterated on, improved on, changed, tailored to the user, by the user working with their user-agent and the LLM choice. One whose failure seems in the realm of personal preference rather than fundamental or harmful or damaging. So what if the English isn't exactly to your liking for the current model? To rest the argument against on such petty matters makes my head reel.

Iterating & improving on the base model & some system prompts allowed in this spec. Rather than using the API to try to cajole this very particular behavior that Jake seems to want everywhere, the user agent could let him set system prompts or pick a more suitable UK trained model, if that's really so important to them. The user agent is the proper channel for the agency Jake is seeking here. Theres nothing preventing the user & their user agent from negotiating what model they use. I don't think we all should be held hostage to naysaying by people who decide that the ability to have "the model needs to talk like a pirate, but this model didn't do that well" decision making. That's circumstantial nonsense, blocking the user agent from being able to work with the user to extend user agency, over such a narrow concern, that must be free to iterate anyhow!!

This API is the best basis we have to allow this negotiation to happen, out of band, outside the scope of the web API offered here, by agents. It's not up to the page to define this in the first place. It's the browser, the user agent that (as always has been the case on the web) builds user agency at its offered level of customizability & complexity. Maybe not every browser offers a "speak like a pirate". That lack is not a ding on the web prompt API!

The objections as stated have no resolution. This is a forever block for all time that is aggreived because not every model is going to behave exactly predictably perfectly. And there's no possible way out of this condundrum. The grievances of this submission are that sites will try to work around this, but the greivance here is built around the assumption that all agency has to lie with the site, that it's the site's obligation to fix US vs British English, that it's the site's need to tailor the agent.

That's not feasible not possible not sensible ever. The user agent is the mediating agent between the site and the user and the agent. That is going to be a complex evolving and dynamic relationship. The "failure" Jake cites here of the site to fully sculpt the experience is unreasonable, is an anti goal. It's up to the user and the browser to shape the agent for them, not each site.

I find these objections to be deeply deeply misguided. But worse, I find them to insist in perfection. There is no direction offered, no improvement suggested. The site can't make agents perfect therefore no one gets agents. That's all this says. It's fucking bullshit and fuck this a lot.

(I love Jake and they have done so much good so so so many times, but this is an impossible situation they are creating while leaving zero space for possibility for maybe and zero leadership for how else we can do what obviously must be done. Alas I think Mozilla at large had become the anti-possible company of web standards, which is a detestable position, one I had hoped might improve, eventually.)

[jaffathecake]

> The user agent is the proper channel for the agency Jake is seeking here. Theres nothing preventing the user & their user agent from negotiating what model they use.

This isn't how it works. As the developer, you use the system prompt to set a particular personality for the chat bot. Eg, when you use an LLM in VSCode, it comes with a system prompt to make it an effective code assistant.

Now, in VSCode, you can select a different model, which is maybe where your misconception comes from. But when you select a different model, it will also use a different system prompt, designed to achieve the same personality, but tailored for that particular model.

Once you figure out why they do that, you'll understand why your position here doesn't make sense.

[jauntywundrkind]

I guess some people don't have access to their system prompt, but that's never been the case for me?

I believe at $LASTJOB even Azure's web GUI let you meddle with that. This isn't cast quite so deeply in stone.

That said I do think it's ok that the app developer ask for British English. My overall concern is more that concern about model behavior & expectations feel very tight & narrow across this, and I don't think the typical well defined rigor and expectations if applied will ever ever ever let LLMs on the web. I think sites see that, I think users see that. Making perfect the enemy of the good is going to doom any attempt, forever, and no progress is possible, if these are the demands. We have to be willing to accept that models are limited, especially small local ones. That's ok.

[ilaksh]

If that was a good argument to not support an LLM feature, then it would be a reason to not add it to any platform API. And yet, it has been added to numerous platforms already.

Different models are just a core aspect of how the technology works.

It's like a canvas can have different possible width and height depending on the device or it's orientation. Or the geolocation API giving more or less accuracy depending on the device. Or Speech Synthesis sounding different depending on the device.

This is really just anti-AI sentiment rather than being constructive.

For now, it needs a permissions UI if it doesn't already have one. And maybe at some point they will add a n IQ level like low, medium, high or something. But developers are going to rely on the specific model 90% of the time anyway if they care about it.

What's going to change is really just that the AI hatred will die down some as people realize how much it helps them, and people will realize not having this feature in Firefox is a failure for personal data autonomy.

And the TOU that are related in Chrome being problematic is an argument FOR Firefox to add this feature, without problematic model terms.

[foolswisdom]

The important part was the following paragraph(s) that explained why this coupling is a compelling problem. It's not the same as just having a platform API.

[nemomarx]

We have different gps reliability per device because they have actual hardware doing that.

Why exactly couldn't models, iq levels, tuning and system prompts be interchangeable in an API for this? Why not let users and devs pick which model to bring or point to one they're paying for, or what have you?

I don't see a world where 90 percent of users of this API pick the same underlying model. It doesn't seem like there's any kind of centralization with ai like that yet.

[ilaksh]

And I didn't suggest they would necessarily select the same model.