[majormajor]

I think there's two camps of think-piece authors emerging - the ones who've tried a bunch of different examples of things and got passable-to-great-looking results; and the ones going deeper into specific areas and hitting the wall in terms of expertise and specificity. Using the GPT 4 API, I'm definitely often hitting limitations, especially around depth of information, and having to "prompt engineer" my way around them. After using a dozen prompt variants to try to prod it in the direction I want without seeing it reflect those changes, a bit of the magic wears off.

I'm bearish on the idea of long-term prompt engineering being a big skillset since I imagine the "understanding the prompt" side of the tools will get better, but I don't see it necessarily getting around the need for specificity of input. It feels like writing a task ticket and giving it to a junior person - what you get back might not be what you need, and a lot of time the true difficulty is knowing exactly what you need up front. Reducing that cycle time is wonderful, but doesn't replace the hard earned skills of knowing what to make.

[fvdessen]

I am in the camp of people who see the current limitations but also see the rate of progress and think those limits may not stand for long. It is yet unclear if it will progress like autonomous driving or like playing go …

[CuriouslyC]

Prompt engineer won't go away, it'll get more "engineer"-like. Knowing how to describe a point in a model's latent space for the generation you want is here to stay, but the black magic and art aspect of it will go away.

For example, in stable diffusion land, lots of people have intuition about the relationship between certain prompts and the output they produce. That intuition is embedding and training data specific, so it's not really transferrable (even to different fine tuned models for stable diffusion 1.5). However, I use clip interrogation to map the portions of the latent that my prompt is pointing to, evaluate the embedding text to find desirable/undesirable elements, then adjust the prompt or add negative prompts to navigate my generations towards what I want.

[aiappreciator]

Prompt engineering is merely the entry-drug to AI-wrangling.

SD has gotten to the point that someone can fine tune a model (LORAs) with 2 days of time and $2 of GPU time.

There'll be roles for AI wranglers in every large company, where you'll be gathering the dataset and building LORA plugins for the AI to adapt specifically for your codebase/customerbase/documentation etc.

There's also processes involved in building APIs for the AI (AIPI?) to use and interface with your documentation and systems, setting up vector databases, monitoring AI output etc.

People who think there won't be job for expert AI users are just coping. Thinking "haha AI will kill your job too". The steam engine was more powerful than 100 men. In the end it required like 30 people up and down the value chain to support the engines, from coal mining, to coal shoving, to maintenance, to manufacturing.

[majormajor]

I'm not sure most codebases are unique enough for that. There will certainly be some of that at places that are doing new things, but for the average online service backend or frontend app programming tasks, I think things like Copilot will see enough and get trained well enough out of the box to be pretty one-size-fits-all.

There will be a lot of business pressure towards using the "good enough" out of the box ones too. If you've got a team of less than a hundred people, rolling your own "datasets, LORA plugins, APIs for AI, vector databases, monitoring, etc" is a multi-person team and significant chunk of new expense. So is the incremental gain their for small to medium teams with relatively "standard" problems?

Kinda like self-hosting at that scale vs using a cloud vendor.

[whatshisface]

"Language model, write me a python script to finetune a language model."

[moonchrome]

I agree with this, for me scenarios where I know what I want are better handled by copilot - I'm way better at writing code than gpt prompts - copilot then picks up the boilerplate as I go along - and since I know what I want it's easy to fact check.

There are some scenarios where it would be useful to have chat like interface in editor to prototype fast - hopefully copilot x delivers.

[simonw]

I think there are two camps of authors using. The ones who tried a bunch of different examples and got terrible results and wrote the whole thing off, and the ones who pushed on through that, kept exploring the capabilities of the model and couldn't believe how useful it could be once they figured out how best to use it.