[ToucanLoucan]

I am dreadfully curious what use cases you're envisioning where a fast, bad/wrong answer is better for anything than a correct, slower answer.

Like hell, if that's the standard, I'll be ChatGPT. You won't need an outrageous graphics card to ask me a question and I'll get you an answer right away. It'll almost certainly be the wrong answer, and is just an ass-pulled guess, but if that's all you want, I'll setup a chat website for myself and start taking queries today. Then investors can give me 10 billion dollars.

[yid]

Here's my pet example...feel free to google around yourself on this.

Problem: I want an AWS CLI command line that requests a whole bunch of wildcard certificates from AWS Certificate Manager (ACM) for a TLD.

Ostensible solution: the AWS official docs have a small snippet to achieve this, BUT -- the snippet on the official page is inadvisable as it leads to a browser cert warning.

So I (skeptically) asked ChatGPT for a command line to achieve what I was trying to do.

Try 1: got basically the snippet from the AWS official docs (but with the inadvisable flag set to the _Correct_ value, strangely)

Prompt 2: please give me more best practice options

Try 2: get back a bunch of new CLI options and their meanings. 3 are useful. 1 is hallucinated. 1 is deprecated.

Prompt 3: keep going with more options

Try 3: 2 more useful new options, 2 more options I chose not to use

As a skeptic, the overall experience was much more efficient that googling around or even reading a manpage. I put it all on the fact that context is maintained between questions, so you don't have to repeat yourself when asking for clarifications.

[nonameiguess]

I'm kind of surprised this worked. Did you actually use the command you ended up with? I'm not even surprised because I think ChatGPT can't figure this out in principle, but because the data itself is poisoned. The top link on every web search I've ever used to AWS CLI commands is to documentation for v1, but v1 has been deprecated for years and the page usually begins with a statement telling you not to use it. Amazon's problem is they never remove old documentation from the web, so 90% of what you find for any given service is no longer correct.

[TeMPOraL]

> and the page usually begins with a statement telling you not to use it

This might be a big part of why GP's case works. The model (GPT-4) most likely understands the concept of documentation being deprecated, so the more often v1 docs say it, the stronger a semantic link between current and obsolete docs, and the more likely it is for ChatGPT to give you answer based on non-deprecated docs.

[yid]

> Did you actually use the command you ended up with?

Yes! Note that I had to use my domain knowledge to sift through the options and eliminate the garbage, but the experience was just _faster_ than repeated searches and digging through ad-laden garbage sites.

[airstrike]

this. in my experience, GPT-4 really shines for groking AWS commands, writing JavaScript code and helping me understand some errors when compiling my terrible Rust code

[nunez]

But what if you enjoy the hunt and building this by hand?

[verdagon]

Sometimes, having a quick, inaccurate, but easily-verifiable answer is better. For example, when you're trying to remember the name of that one function to call, or ideas on where to travel next month.

Also, not super relevant, but in e.g. combat situations one is often better off running now in any direction rather than pondering which direction is absolutely optimal for running from the lion. You'll know soon enough whether it was the right direction. There's probably a metaphor somewhere in there.

[empath-nirvana]

ChatGPT doesn't "almost certainly" give you a wrong answer (especially if you're not asking it math problems). The reason that it hallucinating sometimes is so bad is that it happens _rarely_. If it happened all the time, you'd never use it or trust it. It's just that it happens _enough_ that it's annoying to have to double check everything.

[latexr]

> It's just that it happens _enough_ that it's annoying to have to double check everything.

If you “have to double check everything”, what’s the point? Skip the AI and do the check you were going to do anyway.

[CamperBob2]

I am dreadfully curious what use cases you're envisioning where a fast, bad/wrong answer is better for anything than a correct, slower answer.

Almost anything related to software development. Any answer I get online, whether from Wikipedia or a Github search or a Stack Overflow question or anywhere else, will require careful study and adaptation before I can use it. There will inevitably be things about any given solution that don't apply to whatever I'm doing, or that will be out-and-out wrong. But does that mean I'd be better off without doing a search at all? Of course not.

Same with AI. It can point me in the right direction and save me a lot of trouble, but it can't (yet) do my job for me.

When it gets 10x better -- and I'm sure it will -- then that last part can be expected to change. Which is awesome.

Meanwhile, Stack Overflow and Wikipedia and Github aren't going to get 10x better, ever. Not without cross-pollinating with AI.

[Prickle]

I have an example where I used the Phind model and GPT4 in a mix. The goal was to get multiple iterations of the same code, written in different ways that I could iterate on.

I had this really annoying requirement to get multiple incompatible status codes, and output them in a consolidated human legible way.

So pretend I am using MSSQL, and I have 3 status code tables.

The status code integers are all slightly different. NEW is always 1, but "Completed" could be 5, 7 or 21 depending on table.

The actual "text" is an integer that can be linked to a Text table via the ID and Language name. But, due to the nuances being slightly different, each version can have a different ID. I can't just use DISTINCT on the text ids.

This is even more true when slight differences like "Complete" and "Completed" exist.

So I need to use UNION, to 'combine' multiple unrelated status code tables.

Then I need to get the language text, and SELECT DISTINCT (or something similar) per language.

Then that needs to be outputted as a drop down for the user.

Then, I have to use that as a other, separate input for another SQL query.

To say "using fast, slightly inaccurate GPT code" was faster than doing it by hand would be an understatement.

It gave me about 15 iterations, where about 6 sort of worked. Then I figures it out myself from there.

[radlad]

> I am dreadfully curious what use cases you're envisioning where a fast, bad/wrong answer is better for anything than a correct, slower answer.

>> For some people - I am reluctant to say "for some use cases" - that's very appealing.

You're preaching to the choir.

However, do keep in mind that even authoritative sources found during your own research may be inaccurate. And for some questions, which answer is "right" or "wrong" may not be black and white.

[ToucanLoucan]

> However, do keep in mind that even authoritative sources found during your own research may be inaccurate. And for some questions, which answer is "right" or "wrong" may not be black and white.

I mean, sure. But again: that's just my point restated. What is this doing that a standard search engine does not?

Like, I put shit in my phone's calendar and set reminders so that I don't to think about it anymore. That is a cognitive load (remembering my dentist appointment) that I have now offloaded to technology. And that's useful as all hell, which is why my phone's calendar is full to the tits of everything one would put in a calendar. Now I don't need to think about it. I get messages from my phone when events are coming up, and I get a literal calendar on my screen when I want it, showing me all these things with perfect accuracy.

What is BingGPT in this scenario offloading? It's just a search engine but slower. It doesn't understand what good software is, so it can't make value based judgements on which to recommend. It doesn't know what reliable sources are, and can't evaluate for them, so every bit of information you get back must be treated with a grain of salt. It (probably) doesn't even remotely conceive of why you are asking it a thing or what a good answer to that query would look like because it doesn't know you, it just knows a massive, incomprehensible amount of averages about a ton of things that might be what you want.

And like, that's fine, search engines have had these limitations for my entire life. That's why I'm saying, I don't understand why this is better. It's the same thing as Bing, but slower, and in a chat box.

[radlad]

> I mean, sure. But again: that's just my point restated. What is this doing that a standard search engine does not?

And again, it is quicker than clicking multiple links and can generalize / contextualize what it finds, mapping it to the answer you're looking for.

Have you tried asking one of these tools to write some simple scripts for you? It works decently, actually.

If you didn't already know how to program, this could save you a TON of time, even if it doesn't work perfectly on the first try.

[TeMPOraL]

> If you didn't already know how to program, this could save you a TON of time, even if it doesn't work perfectly on the first try.

Nice thing is, if it doesn't work perfectly on the first try, you can describe the problem (or paste the whole output, errors included), and get back a fixed version that's likely to work this time around.