[anvuong]

It's like saying:

- winners in the 30s were the last time we have pure human to win (before computer)

- winners in the 70s were the last time we have pure human to win (before internet)

- winners in the 90s were the last time we have pure human to win (before search engine)

Why can't we treat LLMs as just another tool like computers, search engines, computing libraries? Why do people keep trying to anthropomorphizing these binaries?

People in the 1800s used to win awards and acclamation by simply hand-cranking numbers for popular calculations (Pi, error functions, etc.) and printing them in a book. This will just be the same thing.

[falcor84]

But it's not the same thing. I went through this conversation between Terry Tao and ChatGPT about the Jacobian Conjecture counterexample [0] and it looks a lot more like a conversation between peers than him using a tool.

[0] https://news.ycombinator.com/item?id=49010345

[anvuong]

Well, how many humans do you think are able to prompt this to the LLM?

"The homogeneity in x is an intertwining between a dilation (x,r,u) to (lambda x, r,u) and a dilation (P,Q,R) to (lambda^-2 P, lambda^-1 Q, lambda R) which seems to collapse the 3d jacobian to a sort of twisted 2d jacobian. Is there a general theory of such twisted jacobians and do you have any sense why those particular dilation weights were used?"

"I can see why the five-dimensional Jacobian has a nice monomial form in rho. Why does this make the three-dimensional Jacobian after restricting to c_2 = rho = 1 and eliminating the delta, eps variables also a monomial (now in x)? Is there some block-diagonal structure or something in the 5D Hessian that allows for a nice reduction? I would have expected some sort of Schur's complement type operation to appear."

I, and probably most people on here, won't be able to get the LLM to write such a detailed conversation, because we are not experts in this field. They are tools.

[bananaflag]

Read these 4 prompts and tell me again how in the future we shall still need human expertise:

https://chatgpt.com/share/6a60b2eb-0b64-83ee-9c76-7931ca1de0...

[butlike]

Because the 4 prompts were actually still prompting for the original prompt (ugh.) in that the computer did not come up with a "complete unconditional counterexample" and the human expertise is required to discern that.

[ivell]

There was an article on Rudi Mandl who pestered Einstein into working on gravitational lensing.

This happened because Rudi was persistent and Einstein was kind enough.

In future, citizen scientists have a chance to work on their ideas using AI, eventhough they don't have the deep domain skills. Of course, an expert would still need to review it as usual. But it is a useful tool to democratize science further.

https://www.sciencenews.org/blog/context/amateur-who-helped-...

[falcor84]

I don't get the argument. It sounds to me like you're saying that because my own chess skills are insufficient to allow me to have a proper game with a grandmaster and I end up losing on move 10 because after a stupid blunder, then I should conclude that this grandmaster is just a tool?

[applfanboysbgon]

"Looks like" being the operative keyword there. Do you feel like you're having a conversation with a peer when you prompt an LLM in the topic you're an expert of? For the love of God, I'd hope not. The whole point is that, even though these things are really good at generating what looks like human output, they are still just regular software algorithms.

[falcor84]

> Do you feel like you're having a conversation with a peer when you prompt an LLM in the topic you're an expert of? For the love of God, I'd hope not.

Yes, I do feel that. Make of it what you will.

[applfanboysbgon]

The only thing I can think is ... how?

If I were to anthromorphize my experience with frontier models, it would be as a mentally challenged child with complete memorization of an encyclopedia and thesaurus. It has the ability to rapidly experiment and potentially succeed at tasks through trial-and-error, but not without constantly corralling it in the correct direction because it would stick a fork in an outlet if unattended for five minutes.

Tao's chat certainly doesn't give me a vibe of talking with a peer. Do you much often have conversations with colleagues where you write one sentence and then get five pages dumped on you, repeating ad infinitum? LLMs can be useful for rubber ducking, and sometimes the plausibly-related word-soup it generates so quickly will help your thinking along faster, but that's not the same thing as a genuine conversation. And it mostly looked like Tao was using it as an advanced calculator, firing off his own ideas for it to quickly do calculations on. I don't know why we need to anthromorphize these tools just because they generate sentences.

[CamperBob2]

If I were to anthromorphize my experience with frontier models, it would be as a mentally challenged child with complete memorization of an encyclopedia and thesaurus.

Memorizing an encyclopedia is not going to help you solve open high-level math problems, is it?

[applfanboysbgon]

> It has the ability to rapidly experiment and potentially succeed at tasks through trial-and-error, but not without constantly corralling it in the correct direction

It kind of could, given the above. Part of an LLM's advantage is that, much like a calculator or a Chess engine, it can iterate over a finite problem space far, far faster than a human can. That much is expected of a useful computing tool.

It is worth noting that we know literally nothing about how the counterexample was achieved. Technically speaking, the person who tweeted it could have solved it themselves with zero LLM assistance and then attributed it to Fable to boost their IPO and ensuing payday. I'm not saying that's actually what happened, but it's hard to draw conclusions without any transparency about the degree of human involvement.

My daily experience certainly does not reflect that of prompting a superhuman intelligence when it routinely flubs commands and destructively drops the PATH of its vm, or bypasses an instruction about passing tests by burning millions of tokens constructing a completely new test suite that rubberstamps its own work when it can't pass the real tests.

[azan_]

If you say "find some unsolved graph theory problem and counterexample for it" and LLM actually does it, is it really you that solved the problem? That's the difference vs other tools.

[fn-mote]

Is this a purely hypothetical question?

Or are you saying that’s what happened in this case. Because that’s not the way I understand it.

[azan_]

It's not, it's basically something that happened. One unsolved problem was solved with the only human input being telling LLM to work harder and to actually solve it after few failed attempts.

[mofeien]

Transcript, four prompts:

https://chatgpt.com/share/6a60b2eb-0b64-83ee-9c76-7931ca1de0...

[netvarun]

https://x.com/Qiaoqiao2001/status/2080003441821163958

While there was a bunch of human effort - it is not that hard to imagine this whole pipeline becoming fully autonomous in the coming months.