[basch]

I think this is mistaking the current .01 iteration with what the technology will be able to achieve. All sorts of groundbreaking technology looks like a minor improvement over the previously refined version until it gets implemented in a way that takes advantage of its strengths, as opposed to just being plugged into old workflows.

LLMs cannot be judged by their first few incarnations. What can be trained into them currently exceeds imagination. Imagination is our limiting factor.

And I don’t say that from the context of “I jumped on the hype train at the end of last year”. I remember reading the 2017 Google transformer paper and thinking “whoa, this is really happening.” The fact it happened in only 5 years is pretty impressive. Im not sure many papers or innovations got my mind spinning quite like that one.

[cornholio]

But there is an unanswered question of how far this technology can go based on its fundamentals. Coding is much like driving, you can't do 80% and let the human do the final 20%, because that final 20% requires reasoning about a well understood design that was implemented throughout the first 80%.

If your fancy AI coder thingy can't really reason about the end task that the code is solving - and there is little to indicate that it does, or that, any moment now, technology will advance to the point that it will - then the 80% will be crap and there exists no human that can finish the last 20%, not even if they put up 200% of the effort required. We still don't have a working AI solution for driving, a well understood and very limited problem domain, never-mind the infinite domain of all problems that can be explained in natural language and solved with software.

What you end up with is a fancier autocomplete, not an AI coder. Boilerplate and coder output might simply increase to take advantage of the new more productive way of generating source code, just like they did for the last decades whenever there was a "revolutionary" new tech, like high level languages, source control, IDEs and debuggers, component distribution etc. etc.

[basch]

You’re already limiting your imagination to “coding.”

These are data transformers that can transform raw data without coding at all. At what point does a model itself replace code?

It’s sort of like a CPU, right. You can have hardware that specialized, or general purpose hardware that can do anything once instructed. LLMs have the ability to be general purpose data manipulators without first having to be designed (or coded) to perform a task.

[pjc50]

> data transformers that can transform raw data without coding at all

How do you know this is 100% reliable, per upthread discussion?

We've already had this problem with Excel in various sciences, which while deterministic has all sorts of surprising behaviors. Genes had to be renamed in order to stop Excel from mangling them: https://www.progress.org.uk/human-genes-renamed-as-microsoft...

AI promises "easier than Excel, but not deterministic". So more people are going to use it to get less reliable results.

[debatem1]

Weird argument. Excel is one of the most popular and profitable programs of all time. If your argument is that LLMs are like Excel, the logical conclusion would be that they would be wildly successful.

[pjc50]

Quite possibly. But not 100% reliable.

[Idiot_in_Vain]

And humans are of course 100% reliable...

[basch]

Isn’t it deterministic with the temperature turned down? You can control when it gives a precise vs fuzzy answer.

[basch]

I didn’t say “LLMs solve all problems” or “there will be no place anywhere for code anymore.”

[cmcconomy]

okay - how do you distinguish between scenarios where it's appropriate and where it's dangerous?

[Footkerchief]

There are two contexts in my experience where it's been important to get the numbers exactly right:

1. Cherrypicking sports statistics for newly set records and the like (NB: this is not lucrative)

2. Financial transaction processing

In most other contexts, especially analytics and reporting, nobody cares and nobody is going to check your math, because the consumers are just trying to put a veneer of numeracy on their instincts.

[cornholio]

Ok, but then you completely give up the ability for human actors to understand and fine-tune the process. It would necessarily be a stochastic product: we don't know exactly how it works, it seems to output correct results in our testing but we can't guarantee it won't cook your dog in the microwave.

[Vanclief]

I completely agree that groundbreaking technologies come from an iterative process. However in the case of LLMs I believe we are already at a point where we can judge where the technology is going as its not the first iteration. Sure it will keep getting better and I think that its already a very useful tool.

My problem with it is that they are over hyping its capabilities and trying to market it as "it makes developers 55% faster" because it writes the code for them. I think it would be a better approach to market it as a great tool for automating repetitive tasks and a better way to consume documentation.

[safety1st]

How would you respond to the central premise of the article? Which I understood as:

* There may not be a lot of differentiation between different LLMs in the long run

* Where there is differentiation, is in data (both the data used to train it and the data provided within its context window for a given query)

* Ergo marrying search to the LLM, while currently in its infancy, will be a big deal and a big differentiator -- because if you can quickly find the right data to pack into the context window, you will get much better results than what we're seeing today.