[asdff]

>> You no longer need to review the code. Or instruct the model at the level of files or functions. You can test behaviors instead.

I think this is where things will ultimately head. You generate random code, purely random in raw machine readable binary, and simply evaluate a behavior. Most random generated code will not work. some, however, will work. and within that working code, some will be far faster and this is the code that is used.

No different than what a geneticist might do evaluating generated mutants for favorable traits. Knowledge of the exact genes or pathways involved is not even required, one can still select among desired traits and therefore select for that best fit mechanism without even knowing it exists.

[hollowturtle]

Why should we throw away decades of development in determistic algorithms? Why tech people mentions "geneticists"? I would never select an algorithm with a "good" flying trait for making an airplane works, that's nuts

[asdff]

But you have selected an algorithm with a "good" flying trait already for making airplanes. Just with another avenue to get to it versus pure random generation. The evolution of the bird has came up with another algorithm for example, where they use flapping wings instead of thrust from engines. Even among airplane development, a lot was learned by studying birds, which are the result of a random walk algorithm.

[hollowturtle]

No there is no selection and no traits to pick, it's the culmination of research and human engineering. An airplan is a complex system that needs serious engineering. You can study birs but up till a certain point, if you like it go doing bird watching, but it's everything except engineering

[asdff]

>it's the culmination of research and human engineering.

And how is this different than the process of natural selection? More fit ideas win out relative to less fit and are iterated upon.

[hollowturtle]

First of all natural selection doesn't happen per se, nor is controlled by some inherent mechanism, it's the by product of many factors external and internal. So the comparison is just wrong. Human engineering is an interative process not a selection. And if we want to call it selection, even though it is a stretch, we're controlling it, we the master of puppets, natural selection is anything but a controlled process. We don't select a more resistant wing, we engineer the wing with a high bending tolerance, again it's an iterative process

[asdff]

We do select for a more resistant wing. How did we determine that this wing is more resistant? We modeled its bending tolerance and selected this particular design against other designs that had worse evaluated results for bending tolerance.

[xg15]

By that logic, everything humans do is per definition result of natural selection. Everything is a sphere if you zoom out far enough.

However your starting definition was more limited. it was specifically about "creating candidates at random, then just picking the one that performs best" - and that's definitely not how airplanes are designed.

(It's not even how LLMs work, in fact)

[phist_mcgee]

Great rule of business: sell a solution that causes more problems, requiring the purchase of more solutions.

[hollowturtle]

Customers are tired of getting piles of shit, look at the Windows situation

[lenerdenator]

Or don't sell the solution. When you have monopolies, regulatory capture, and endless mountains of money, you can more or less do what you'd like.

[hollowturtle]

That's a lie, people will eventually find a way out, it was always like that, being it open source or by innovating and eventually leave the unable to innovate tech giants dying. We have Linux and this year will be the most exciting for the Linux desktop given how bad the Windows situation is

[bradlys]

Only been hearing that for twenty years and these tech giants are bigger than they’ve ever been.

I remember when people said Open Office was going to be the default because it was open source, etc etc etc. It never happened. Got forked. Still irrelevant.

[stavros]

Maybe, but you won't be able to test all behaviors and you won't have enough time to try a million alternatives. Just because of the number of possibilities, it'll be faster to just read the code.

[asdff]

Eventually the generation and evaluation will be quite fast where testing a million alternatives will be viable. Impressive you suggest that there might be a million alternatives but it would be faster to just read the code and settle on one. How might that be determined? Did the author who wrote the standard library really come up with the best way when writing those functions? Or did they come up with something that seemed alright to ship relative to other ideas people came up with?

I think we need to think outside the box here and realize ideas can be generated, evaluated, and settled upon far faster than any human operates. The idea of doing what a trillion humans evaluating different functions can do is actually realistic with the path of our present technology. We are at the cusp of some very remarkable times, even more remarkable than the innovations of the past 200 years, should we make progress on this effort.

[jimbokun]

This comment strikes me as not having a good intuition for how fast the space of possible programs can grow.

[asdff]

You don't think the space of possible problems can be parsed with increased compute?

[xg15]

Not for all problems, definitly not. As an example of extremely fast-growing problem spaces, look at the Busy Beaver functions:

https://en.wikipedia.org/wiki/Busy_beaver

[lefra]

A million alternatives is peanuts. Restricting the search space to text files with 37 possible symbols (letters, numbers, space), a million different files can be generated with just 4 symbols.

A trillion is 8 symbols. You still haven't reached the end of your first import statement.

I just took a random source file on my computer. It has about 8000 characters. The number of possible files with 8000 characters has 12500 digits.

At this point, restricting the search space to syntactically valid programs (how do you even randomly generate that?) won't make a difference.

[johndough]

> restricting the search space to syntactically valid programs (how do you even randomly generate that?)

By using a grammar. Here is an example on how to only generate valid JSON with llama.cpp: https://github.com/ggml-org/llama.cpp/blob/master/grammars/R...

> A trillion is 8 symbols. You still haven't reached the end of your first import statement.

Since LLMs use tokens from a vocabulary instead of characters, the number is likely somewhere in the lower billions for the first import statement.

But of course, LLMs do not sample from a uniform random distribution, so there are even fewer likely possibilities.

[stavros]

If this were viable, we'd all be running Haiku ten times in the time it took to run Opus once, but nobody does.

[asdff]

We don't have the compute for this today. We will in several centuries, if compute growth continues.

[hiimkeks]

We definitely would need centuries for this, because Moore's law has been dead for a while, while the number of possible programs grows exponentially in it's length.

But I hope we have more efficient ways to do this in a century.

[bandrami]

That's why you buy a quantum computer. It writes every possible piece of software at once and leaves you the trivial task of curating the one you want.

[stavros]

Very much like what our junior developers do!

[formerly_proven]

This is a recurring fantasy in LLM threads but makes little sense. Writing machine code is very difficult (even writing byte code for simple VMs is annoying and error-prone). Abstractions are beneficial and increase productivity (per human, per token). It makes essentially no sense to throw away seven decades of productivity increasing technologies to have neural nets punch cards again, and it's not going to happen unless tokens become unimaginably cheap.

[asdff]

Compute is always increasing on this planet. It makes no sense to stick with seven decade old paradigms from the time when we were simply transmuting mathematical proofs into computational functions. We should be exploring the void, we will have the compute for this. Randomness will take away the difficulty as we increase compute to parse over these random functions in reasonable time frames. The idea of limiting technological development to what our ape brain can conceive of on its own on human biological timescales is quite a shackle, honestly.

[lunar_mycroft]

For this to work, you'd have to fully specify the behavior of your program in the tests. Put another way, at that point your tests are the program. So the question is, which is a more convenient way to specify the behavior of a program: a traditional programming language, or tests written in that language. I think the answer should be fairly obvious.

[asdff]

Behavior does not need to be fully specified at the outset. It could be evaluated after the run. We've actually done this before in our own technology. We studied birds and their flight characteristics, and took lessons from that for airplane development. What is a bird but the output of a random walk algorithm selected by constraints bound by so many latent factors we might never fully grasp?

[lunar_mycroft]

> Behavior does not need to be fully specified at the outset. It could be evaluated after the run.

This doesn't work when the software in question is written by competent humans, let alone the sort of random process you describe. A run of the software only tells you the behavior of the software for a given input, it doesn't tell you all possible behaviors of the software. "I ran the code and the output looked good" is no where near sufficient.

> We've actually done this before in our own technology. We studied birds and their flight characteristics, and took lessons from that for airplane development.

There is a vast chasm between "bioinspiration is sometimes a good technique" and "genetic algorithms are a viable replacement for writing code".

[asdff]

Genetic algorithms created our species, which are far more complex than anything we have written in computer science. I think they have stood up to the tests of creating a viable product for a given behavior.

And with future compute, you will be able to evaluate behavior across an entire range of inputs for countless putative functions. There will be a time when none of this is compute bound. It is today, but in three centuries or more?

[suddenlybananas]

https://www.mcgill.ca/oss/article/student-contributors-did-y...

Evolution ain't all that great.

[troupo]

> Genetic algorithms created our species, which are far more complex than anything we have written in computer science. I think they have stood up to the tests of creating a viable product for a given behavior.

Yes, and our species is a fragile barely functioning machinery with an insane number of failing points, and hillariously bad and inefficiently placed components.

[mitemte]

I can see a lot of negatives in relation to removing the human readable aspect of software development. Thorough testing would be virtually impossible because we’d be relying on fuzzing to iron out potential edge cases or bugs.

In this situation, AI companies are incentivised to host the services their tooling generates. If we don't get source code, it is much easier for them to justify not sharing it. Plus, who is to say the machine code even works on consumer hardware anyway? It leads to a future where users specify inputs while companies generate programs and handle execution. Everything becomes a black box. No thank you.

[asdff]

All these questions are true for agriculture, yet you say "yes thank you, and please continue" for that industry I am sure, which seeks to improve product through random walk and unknown mechanisms. Maybe take a step back and examine your own biases.

[troupo]

> All these questions are true for agriculture, yet you say "yes thank you, and please continue" for that industry I am sure, which seeks to improve product through random walk and unknown mechanisms.

Tell me you know nothing about modern agriculture without telling me that

[bossyTeacher]

> ou generate random code, purely random in raw machine readable binary, and simply evaluate a behavior. Most random generated code will not work. some, however, will work. and within that working code, some will be far faster and this is the code that is used.

Humans are expensive but this approach seems incredibly inefficient and expensive. Even a junior can make steady progress against implementing a function, with your approach, just monkey coding like that could take you ages to write a single function. Estimates in software are already bad, they will get worse with your approach

[asdff]

Today it might not work given what a junior could do against the cost of compute through random walk, but can you say the same in three centuries? We increase compute by the year, but our own brainpower does not increase on those terms. Estimates are that we are actually losing brainpower over time.

[jimbohn]

The problem is that programming logic/state is discrete and not continous so you can't assume similar behaviour given "similar state", and that possible states grow exponentially. Selecting the desired state will mean writing an extremely detailed spec that is akin to a programming language, which is what Dijkstra hinted at in the past.

[asdev]

all fun and games until you need to debug the rats nest that you've been continually building. I am actually shocked people who have coded before have been one-shotted into believing this

[asdff]

If a bug rears its head it can be dealt with. Again, this is essentially already practiced by humans through breeding programs. Bugs have come up, such as deleterious traits, and we have either engineered solutions to get around them or worked to purge the alleles behind the traits from populations under study. Nothing is ever bug free. The question is if the bugs are show stoppers or not. And random walk iteration can produce more solutions that might get around those bugs.

[SilenN]

Agree! Don't let it get too bad. See my other comment on debugging and refactoring being a core part of your workflow.

[madrox]

You're describing genetic algorithms: https://en.wikipedia.org/wiki/Genetic_algorithm

[asdff]

Exactly. As compute increases these algorithms will only get more compelling. You can test and evaluate so many more ideas than any human inventors can generate on their own.

[HPsquared]

I suppose you could generate prompts from "genes" somehow.

[Dilettante_]

We would of course need to specify the behaviors to test for. The more precisely we specify these behaviors, the more complexly our end product would be able to behave. We might invent a formal language for writing down these behaviors, and some people might be better at thinking about what kind of tests would need to be written to coax a certain type of end result out of the machine.

But that's future music, forgive a young man for letting his imagination run wild! ;)

[asdff]

If we consider other fields such as biology, behaviors of interest are specified but I'm not sure a formal language is currently being used per say. Data are evaluated on dimensional terms that could be either quantitative or qualitative. meta analysis of some sort might be used to reduce dimensionality to some degree but that usually happens owing to lack of power for higher resolution models.

One big advantage of this future random walk paradigm is you would not be bound by the real world constraints of sample collection of biological data. datasets could be made arbitrarily large and cost to do so will follow an inverse relationship with compute gains.

[themafia]

> You generate random code,

Code derived from a training set is not at all "random."

[asdff]

Not saying training set or present LLM. Truly random binary generator left to its own device. Lets evaluate what spits out from that iterated several trillion times over with the massive compute capability we will have. I am not thinking of this happening in the next couple years, but in the next couple of centuries.

[themafia]

> Truly random binary generator left to its own device.

This has been tried. It tends to overfit to unseen test environment conditions. You will not produce what you intend to produce.

> the massive compute capability we will have

Just burn carbon in the hopes something magical will happen. This is the mentality of a cargo cult.

[asdff]

We also burn carbon to feed the brain. Compute is what is increasing in capability on the scale of orders of magnitudes just within our own lifetimes. Brainpower is not increasing in capability. If you want future capabilities and technological advancement to occur at the fastest pace possible, eventually we have to leave the slow ape brain behind in favor of sources of compute that can evaluate functions several orders of magnitude faster.

[Fire-Dragon-DoL]

How do you handle the larger amounts of tests? I did this but my PRs are larger because more tests are needed

[asdff]

I'm not sure. My thinking is this will occur on the scale of the next several hundred years, not the next few.

[dns_snek]

And how exactly do you foresee probabilistic systems working out in real life? Nobody wants software that seldom does what they expect, and which tends to trend toward desirable behavior over time (where "desirable" behavior is determined by the sum of global feedback and revenue/profit of the company producing it).

Today you send some money to your spouse but it's received by another person with the same name. Tomorrow you order food but your order gets mixed up with someone else's.

Tough luck, the system is probabilistic and you can only hope that the evolutionary pressures influence the behavior to change in desirable ways. This fantasy is a delusion.

[AkshatM]

You're thinking of probabilistic systems at run time. People are talking about probabilistic systems at compile time.

Whatever gets generated, if it passes tests and is observably in compliance with the spec, is accepted and made permanent. It's the clay we're talking about Jackson Pollocking, not the sculpture.

[dns_snek]

I'm not, I'm thinking of poorly engineered systems that display buggy, unintended behaviors at runtime.

> observably in compliance with the spec

That's so easy to say and so incredibly hard to implement! Most unintended behaviors will never end up being prohibited/defined in the specification written by non-programmers.

The act of translating requirements from human language into well-defined semantics is what programming is.

[suddenlybananas]

Tests don't go over the space of all possible inputs and you have no idea how other inputs are generalized or interpolated.

[asdff]

I think you misunderstand. Once established a function found through random walk is no different than a function found in any other way. If it works it works, if it doesn't it doesn't.

[dns_snek]

I didn't misunderstand. I'm talking about all the exciting unintended behaviors you're adding and all the invariants that you're not preserving by arriving at solutions through randomness rather than careful design and engineering.

Yes a function will always do "the exact same thing" at runtime, but that "thing" isn't guaranteed to be free from race conditions and other types of bugs.

[lenerdenator]

I can tell you why this won't go this way:

Customers.

When you sell them a technological solution to their problem, they expect it to work. When it doesn't, someone needs to be responsible for it.

Now, maybe I'm wrong, but I don't see any of the current AI leaders being like, "Yeah, you're right, this solution didn't meet your customer's needs, and we'll eat the resulting costs." They didn't get to be "thought leaders" in the current iteration of Silicon Valley by taking responsibility for things that got broken, not at all.

So that means you will need to take responsibility for it, and how can you make that work as a business model? Well, you pay someone - a human - who knows what they're looking at to review at least some of the code that the AI generates.

Will some of that be AI-aided? Of course. Can you make a lot of the guesswork go away by saying "use commonly-accepted design patterns" in your CLAUDE.md? Sure. But you'll still need someone to enforce it and take responsibility at the end of the day if it screws up.

[asdff]

You are thinking in terms of the next few years not the next few centuries. Plenty of software sold today fails to meet expectations and no one eats costs.