[3eb7988a1663]

What does effective use look like? I have attempted messing around with a couple of options, but was always disappointed with the output. How do you properly present a problem to a LLM? Requiring an ongoing conversation feels like a tech priest praying to the machine spirit.

[Guthur]

In my opinion that's literally what we're aiming for, whether or not intentionally.

[CSMastermind]

Candidates generally use it in one of two ways: either as an advanced autocomplete or like a search engine.

They'll type in things like, "C# read JSON from file"

As opposed to something like:

> I'm working on a software system where ... are represented as arrays of JSON objects with the following properties:

> ...

> I have a file called ... that contains an array of these objects ...

> I have ... installed locally with a new project open and I want to ... How can I do this?

No current LLMs can solve any of the problems we give them so pasting in the raw prompt won't be helpful. But the set up deliberately encourages them to do some basic scaffolding, reading in a file, creating corresponding classes, etc. that an LLM can bang out in about 30 seconds but I've seen candidates spend 30 minutes+ writing it all out themselves instead of just getting the LLM to do it for them.

[neonsunset]

GitHub Copilot Edit can do the second version of this. It is pretty good at it too. It sometimes gets things wrong but for your average code (and candidates typing in "C# read JSON from file" are way below average unless they never written in C#), if you give all the files for a specific self-contained part of the program, it can extend/modify/test/etc. it impressively well for an LLM.

The difference compared to where we were just 1-2 years ago is staggering.

Edit: the above is with Claude-3.5-Sonnet

[codeyperson]

High level - having a discussion with the LLM about different approaches and the tradeoffs between each

Low level - I'll write up the structure of what I want in the form of a set functions with defined inputs and outputs but without the implementation detail. If I care about any specifics with the functions I'll throw some comments in there. And sometimes I'll define the data structures in advance as well.

Once all this is set up it often spits out something that compiles and works first try. And all the context is established so iteration from that point becomes easier.

[feoren]

> High level - having a discussion with the LLM about different approaches and the tradeoffs between each

I honestly can't imagine this. If the AI says "However, a downside of approach B is that it takes O(n^2) time instead of the optimal O(nlog(n))", what do you think the odds are that it literally made up both of those facts? Because I'd be surprised if they were any lower than 30%. It's an extremely confident bullshitter, and you're going to use it to talk about engineering tradeoffs!?

> Once all this is set up it often spits out something that compiles and works first try

I'm sorry, but I'm extremely* doubtful that it actually works in any real sense. The fact that you even use "compiles and works first try" as some sort of metric that the code it's producing shows how easily it could slip in awful braindead bugs without you ever knowing. You run it and it appears to work!? The way to know whether something works -- not first try, but every try -- is to understand every character in the code. If that is your standard -- and it must be -- then isn't the AI just slowing you down?

[sanarothe]

I don't code for a living, and I'm probably worse than a fresh grad would be but I use:

"Please don't generate or rewrite code, I just want to discuss the general approach."

Bc I don't know any design patterns or idiomatic approach, being able to discuss is amazing.

Though quality and consistency of responses is another thing... :)

[arthens]

> I honestly can't imagine this. If the AI says "However, a downside of approach B is that it takes O(n^2) time instead of the optimal O(nlog(n))", what do you think the odds are that it literally made up both of those facts? Because I'd be surprised if they were any lower than 30%. It's an extremely confident bullshitter, and you're going to use it to talk about engineering tradeoffs!?

Being confidently incorrect is not a unique characteristic of AIs, plenty of humans do it too. Being able to spot the bullshit is a core part of the job. If you can't spot the bullshit from AI, I wouldn't trust you to spot the bullshit from a coworker.

[feoren]

But if I have a coworker who bullshits 30% of the time, I get them off my project. Because they too are just slowing everything down.

[concordDance]

It can list tradeoffs and approaches you might have forgotten. Thats the big use case for me.