[logicchains]

Some random team at Berkeley beat them to it by a couple days: https://largeworldmodel.github.io/ . It's just a matter of throwing compute at it, nothing fancy. OpenAI could probably do 1M token context too but they haven't yet found a way to make it profitable (neither have Google; the most they actually offer customers is 256k).

[x86x87]

This brings a very interesting question: if you could have an AI software engineer today but it would cost you 1 trillion dollars, would you want and be able to afford it?

There is a reason why we still have people working at McDonald's even though fully automating it has been possible for a couple of decades now.

[jjmarr]

It's the same reason that for 80 years after the invention of the commercial icemaker in 1842, the American ice-harvesting industry produced more frozen water than manufacturing plants. And the ice trade did not exist until 1806.

https://en.wikipedia.org/wiki/Ice_trade

It was more economical to send people out to cut ice from a lake in Maine and ship it by rail to Chicago than it was to just freeze water from a local supply. It was also more reliable since the technology was mature, versus ice plants that often broke down when meatpackers needed a consistent supply.

There's no reason why this won't be the case for AI unless semiconductor manufacturing continues its exponential performance/cost growth. The demand for technologically obsolete goods and services do not instantly disappear when a superior product enters the market.

Human software engineers right now are more reliable than AIs for most price-points. This is true for most industries in which machine learning is present.

[tarruda]

> cost you 1 trillion dollars

How did you come up with this number? It seems pretty unrealistic.

> There is a reason why we still have people working at McDonald's even though fully automating it has been possible for a couple of decades now.

Maybe the low salary is the reason? If it is a bit more costly to automate certain aspects of manual labor, then the low salaries might remove the incentive to do so. This is not the case for software engineering.

[toyg]

Beyond the snark, this is basically it. It's the same reason the Roman Empire, despite all its technological prowess, never tried hard to automate relatively low-hanging-fruit tasks: because slaves were cheap, plentiful, and more flexible ("reprogrammable") than anything mechanical could ever be.

If it costs $1m p/y to run a machine that cooks burgers and fries, or $30k for an employee who can do that _and_ cover something else when someone else is ill, it's a no-brainer. But businesses had to discover that the hard way; until the 80s, most people were still convinced automation would win everywhere, because it had won (and won big) in manufacturing. A combination of factors, from the '80s onwards, made labor costs effectively fall, which created our reality where certain jobs are so cheap that automating them makes no sense.

The "problem" is that, in certain regions, software development costs reached a point where automation looks very, very appealing. If a machine costs 500k p/y to replace a few 150k p/y SWEs without all those pesky employment complications, businesses will happily choose "AWS AI CloudDeveloper"...

[tarruda]

> If a machine costs 500k p/y

Do you mean an AI programmer would cost $500k per year? If so I think you greatly overestimate the cost.

Recently I did some text processing with GPT-4 turbo (128k context) and I reached the daily limit of 5 million tokens. IIRC it cost me around $70 bucks for the day.

I think $70 is the hourly rate of a SE with $150k salary working 40 hours per week. Note that we are at early stages with this tech, it will probably only get cheaper from here.

[BandButcher]

"IIRC it cost me around $70 bucks for the day."

Sure, for you that was the price. Enterprise cost would be way different.

"Note that we are at early stages with this tech, it will probably only get cheaper from here."

Haha people who pay for these ai tools can only hope...Ask any cloud provider, streaming service, or utility company if their prices are cheaper now than before.

As these ai tools get better, they will require more resources to run (according to altman's 7 trillion dollar request) and most likely drive up the costs.

But hopefully you are right though, as i believe we as humanity would be best served spending as little money and resources as possible on AI.

[toyg]

> If so I think you greatly overestimate the cost.

I suspect you underestimate it. Raw engine cost is one thing; what businesses downstream will actually pay, is another. Look at AWS: a lot of businesses don't even touch it directly, their vendor ISPs do. If "AIDev" really becomes a thing, businesses will buy specialized services (e.g. "ApiBuilder.io", "YAMLCrusher.io", etc etc), which will obviously command a premium on top of top-tier, 5-9s guaranteed, "raw" ml engines.

[calvinmorrison]

Aw look, hn is reinventing capitalism from first principles!

[x86x87]

Fwiw my question was rethorical.

And it was meant to highlight that even if you have the tech (which we don't - the cheap tricks chatgpt or copilot do are impressive but still cheap tricks - are super expensive when it comes to actually training the models) it may not make economic sense to deploy them.

Even if it makes sense to deploy them the social unrest and volatility that will result in society may not end up well. (What's the point if all the consumers go away or they cannot actually buy the shit you're producing)

[jjmarr]

GitHub CoPilot charges $10 for a subscription that it loses an average (!) of $20 a user on.

https://www.theregister.com/2023/10/11/github_ai_copilot_mic...

"Make it profitable" appears a secondary concern in the AI space.

[cosmodisk]

I started using it recently. $30, 50, or even $100 a month is litterly nothing for most companies in wester world. They'll hike up the prices eventually.

[bearjaws]

I believe that is where they are implying they do it without increasing memory utilization dramatically.

If 1M context uses 32x the memory of 32k, its a non-starter. Even a smallish LLM like Mixtral uses 4-8gb of memory just for your prompt. You would have 256+GiB at 1M...

[tarruda]

> It's just a matter of throwing compute at it, nothing fancy.

I read somewhere that there was a recent breakthrough that enabled this.

Even if it costs a lot to run inference with 1M token context, it is hard to imagine it would cost anywhere close to a software engineer salary.