[razcle]

Hey Tombert,

wrt did you read the article? I was quite specific about the ways I think LLMs are blurring the lines. I don't think its true for general engineering but I do think its true for applications being built with LLMs.

Also its still very early

[nexus_six]

It's not early. It has reached a plateau. Are there betting odds for "AI" (LLM) benchmarks somewhere? I will bet money

[tombert]

Reaching a plateau doesn't imply that it's not early. It's still entirely plausible that we come up with a newer better model in ten years that gives us true AGI or runs on cheaper hardware or just gives us a closer approximation to human reasoning.

[razcle]

https://www.metaculus.com/questions/5121/date-of-artificial-...

[nexus_six]

Is there a paid market? This seems to be "play money" or "play points". I can't find anything like Polymarket with similar contracts.

[tombert]

I'm just saying that I don't think I agree with some of this; even if PMs are writing the prompts (and calling that "prompt engineering"), it's not equivalent because they don't know how to audit the code given to them.

A PM might generate that SQL thing I mentioned and just blindly cut and paste it. For any application with more than one user, that is a bug, it's incorrect, and it's not like this is some deep cut: upserts happen all over the place.

I didn't finish the entire article, I disagreed with the line, "Prompting Is Here To Stay and PMs—Not Engineers—Are Going to Do It", because I fundamentally do not think that is true unless AI models get considerably better.

It's possible they will, maybe OpenAI will crack AGI or maybe these models will just get a lot better at figuring out your intent or maybe there's another variable that I'm not thinking of that will fix it.

I hate the term "prompt engineer" because I don't think it's engineering, at least not really. I will agree that there's a skill to getting ChatGPT to give you what you want, I think I'm pretty good at it even, but I hesitate to call it engineering because it lacks a lot of "objectivity". I can come up with a "good" prompt that will 90% of the time give me a good answer, but 10% of the time give me utter rubbish, which doesn't really feel like engineering to me.

I saw the line: `As AI models become able to write complex applications end-to-end, the work of an engineer will increasingly start to resemble that of a product manager.`, and while I don't completely disagree, I also don't completely agree either. Even when I heavily abuse ChatGPT for code generation, it doesn't feel at all like I'm barking orders to a human. It might superficially resemble it but I'm not convinced that it's actually that similar.

I hope I'm not coming off as too much of a dick here, I apologize if I am, and obviously a blog post in which you wax philosophical about the implications of new technology is perfectly fine. I think I'm just a bit on edge with this stuff because you get morons like Zuckerberg claiming they'll be able to replace all their junior and mid level engineers with AI soon, and I think that's ridiculous unless they have access to considerably better models than I do.

[arcsincosin]

My read–which may be wrong–is that much of the article is discussing applications where the end user is interacting with an interface that queries an LLM using baked-in prompts (in one case, a marketing content generation tool). These prompts are being written by the PM. The PM is not writing prompts LLMs to generate code, the PM is writing prompts which are hidden behind a web form or button or something in an interface, hence the prompts being part of the codebase. The author argues that when a PM is editing these prompts they are delivering an artifact that looks more like an engineer's artifact than a PM's artifact, traditionally.

[rsynnott]

> Also its still very early

“Jam tomorrow” will only get you so far.