[bongodongobob]

Maybe the prompt engineering should have caught on more. I'm convinced that the large swaths of people commenting here and elsewhere "I don't get AI, it's just a parrot and it's always wrong and hallucinates, it's not useful" just don't understand that the prompt matters and the idea isn't to one shot everything. It writes good code for me every day, so I can only assume they're asking "Write me an OS from scratch" and then throwing their hands up when it obviously fails.

[lolinder]

I think that calling it "prompt engineering" is what made it fail to catch on. We didn't call it "Google engineering" back in the day when you could actually craft a Google search to turn up useful results, we called it "Google-fu" [0].

"Google-fu" sounds like a fun skill to learn and acquire, where "prompt engineering" sounds either like something well out of reach or like pretentious nonsense depending on the audience.

[0] https://blog.codinghorror.com/google-fu/

[fuzztester]

>"Google-fu" sounds like a fun skill to learn and acquire, where "prompt engineering" sounds either like something well out of reach or like pretentious nonsense depending on the audience.

More likely, "prompt engineering" is a marketing term made up by AI marketroids (cf. androids) hoping to make developers feel better about their reduced roles in this "grand new AI age".

[fzzzy]

Prompt-fu sounds cool

[dvfjsdhgfv]

Arguably you could charge more money from courses on prompt engineering than on prompt-fu.

[chipdart]

> I think that calling it "prompt engineering" is what made it fail to catch on.

I don't think so.

I mean, clearly calling it "engineering" threw some people off, in the same blend as some gatekeepers cringe at calling train drivers "railroad engineers". But that's puerile gatekeeping that misses the whole reason why there is a vast need to know how to "engineer prompts".

The truth of the matter is that the focus of "prompt engineering" is being able to put together inputs that solve business problems in professional settings. You need to have full control over the generative process to integrate it's output in a business setting. That requires specialized knowledge way beyond naive requests expressed in natural language.

Complaining about "prompt engineering" because that only focuses on specifying queries and operating a specific service makes as much sense as complaining about SQL/database/postgres engineering because that only focuses on specifying queries and operating a specific service.

Before trying to dismiss "prompt engineering" through gatekeeping logic, first you need to justify why there is no need to know what you're doing to get outputs by feeding the right inputs. Even in subreddits dedicated to using generative AI to create images and videos,they started to outright ban posts where the contents are posted without the prompts used to create it.

[skywhopper]

To me it’s more like, if I have to carefully craft English language prompts in a conversational back-and-forth to get things done, then I am not really interested in doing that job, which sounds like being a manager or a teacher, and in practice just makes me feel totally dead, sad, and quite frankly bored.

That’s just not an interesting or rewarding way to interact with a computer, and the last thing I want to do is add long wait times and nickel-and-dime cost to the process. Layer on using different LLMs for different tasks or trying them out against each other and cross-checking output and it’s a mind-numbingly indirect way to get anything accomplished that in the end teaches me nothing and develops no useful skill that I enjoy practicing.

If it works for you, great, but even the most honest and genuine fans make it sound like a nightmare to me.

[trog]

Could not agree more. If the utility of this thing is based entirely on the right sequence of magic words why aren't we calling it "prompt wizardry" or something that better encapsulates the nature of it.

[justinclift]

"That's Fred, a prompt magician of the third rank!" ;)

[evilduck]

If the utility of all computers is based on the right sequence of magic words why do we call them software engineers instead of something better like "code wizards" that encapsulates the nature of it?

[trog]

I guess the difference is code is (almost always) totally deterministic. Or at the very least, they're designed so that is a mostly safe assumption.

It doesn't seem likely an LLM will ever do that. Maybe at a certain point of sophistication? But if the model is regularly changing - which they almost all will be, if they're expected to be up-to-date - there is a strong change they'll be different every time they're used.

(I've been getting different behaviour in even relatively narrow ML-based systems for years. Google Assistant is my prime example - I regularly use the phrase "add to my calendar on the 20th of September at 5pm, go to the park". Almost all the time, it works perfectly. But a couple times a year at least, it won't process this as an action - it just does a Google web search for this string.)

[evilduck]

Code can be deterministic if you're only doing trivial things or have very simple systems. Beyond that it only has sufficient determinism to cross a threshold into being considered useful. Dead letter queues, uncaught errors, kernel panics, race conditions, deadlocks, cosmic bit flips, dumb avoidable bad choices from unexpressive languages, malicious actors, resource constraints, and so on; the real world of software we live in is a duct taped together mess of half measures that only mostly do what we want at its best. So much of the work in product programming is handling all the things that can go wrong.

So yeah, prompt "engineering" is indeed a silly term, but software "engineering" kicked off the dilution of that word ages ago. And GPT models can be inspected and measured for input and output, prompts can be analyzed for their effects and usefulness, temperature settings even directly control some degree of determinism. It's not like models change on a whim unless you're just using end user products. Anthropic, Huggingface, AWS, OpenAI, they let you pick a release model version in your API calls and stick with it for a long time. If you're self hosting a fine tuned Llama 70b, nobody will ever force you to update it if you get it doing a task to your expectations. The quality of deterministic behavior in AI is currently lower than that of Excel or C code, but it's also serving a wholly different purpose, people want it to be creative and create novel nondeterministic outputs, comparing them is a bit silly.

[lolinder]

To be fair to what OP is saying, it's not so much that you have to carefully craft the prompts every single time, it's that there is a linguistic register you have to adopt in order to get results out of an LLM. The initial learning process for that register can be hard, but once you've learned it it comes naturally.

I think of it as similar to Googling in the early days. What started as a skill I had to pick up became second nature and I could find things faster than my family without even really thinking about what I was doing. It just became natural.

[lurking_swe]

to be fair, expecting most software engineers, who typically have a bachelors degree, to be able to communicate well in english is not asking for a lot. Via a textual medium no less! But apparently it is…

Most of my colleagues communicate with chatgpt in broken english, or they ask a question while leaving out crucial details about their problem. They’re always surprised when i am able to get a useful response from chatgpt when they couldn’t. it’s comical sometimes.

I 100% hear you on the “not a fun way to interact” though. To each their own. I personally enjoy it, it’s like a rubber duck that can actually talk back. :) not for everyone though.

[LargeWu]

It's not so much that communicating in English is the problem, at least for me. I'm a native English speaker and have a reasonably strong command of the English language. I'm able to craft my words to convey specific tone or meaning.

The problem is that GenAI is a complete black box with nondeterministic outputs. I can write code and I know with a very high degree of confidence what I expect it to do. Asking an LLM or a generative image program for something, I have no idea what it'll give me. It gives no feedback other than results, which may or may not be what I want. If not, I have to reverse engineer what I think it might want me to say in order to get desired results. And the same query placed another time might give a completely different answer. I don't deny that it can do some impressive things given the correct inputs, but I am not inclined to spend my time searching for the magic words.

[chipdart]

> To me it’s more like, if I have to carefully craft English language prompts in a conversational back-and-forth to get things done, then I am not really interested in doing that job, which sounds like being a manager or a teacher, and in practice just makes me feel totally dead, sad, and quite frankly bored.

You're showing a fundamental misunderstanding (or ignorance) of the whole problem domain.

For starters, you place an awful lot of emphasis on what you think is "carefully craft English language prompts". That makes as much sense as characterizing the job of a database engineer as "carefully crafting quasi-English language prompts". The language used is completely irrelevant, and being able to use in some circumstances something resembling natural language to build up context does not take away from it.

Any remotely honest and objective analysis of the topic would start from similar activities, and to start off the areas of work where Llama are being used. For image/video generation you need to look at graphics design, video editing, video production, illustrators, etc. These activities, by their own nature, are iterative and exploratory. Then for text you have the work of copywriters and editors, and even writers and essayisgs. The work is fundamentally iterative and exploratory. Then you have work like exploratory data analysis/statistics/data mining. Every aspect of that work is iterative, even the reporting part.

[lazystar]

to me it sounds like a job that would be similar to a search engine optimization engineer - studying the output of a third-party program when providing that program with different sets of keywords.

[BlackFly]

If it only solves the problems I already find trivial then it is a parrot. Nowadays we all have a calculator with us but if you emphasize that fact and choose to not practice and excel at basic arithmetic then you will be unable to perform higher mathematics that require it at every step. Of course if your problem is always already a solved problem then sure a parrot can be convinced to spit it out.

So yes, the actual question for software engineering would be how to get AI to produce and iterate on an OS. The hallucinations aren't the only problem then, the lack of predictability in the answers is the biggest issue.

[refulgentis]

Been quietly wondering something similar to you for a year: I've ended up 95% confident that phenomena is due to people evaluating it in terms of "does it replace me?"

Cosign prompt engineering. My startup is tl;dr "what if i made a on-every-platform app that can sync and let you choose whatever ai provider, and you pay at cost. and then give you a simple UI for piecing together steps like dialogue / ai chat / search / retrieve / use files"

Seems to me the bigs are completely off the mark, lets cede the idea there's an omniscient AI available. Literally right now.

Cool.

It still has no idea how you work.

you could see 42, in hitchhiker's guide the galaxy, as a deep parody of this category error