Hacker News new | ask | show | jobs
by nybsjytm 715 days ago
Great article, covers well both the achievements and the shortcomings. It's crazy how many people write about these kinds of AI developments while completely skipping over anything like the following:

> The “good news is that when AlphaFold thinks that it’s right, it often is very right,” Adams said. “When it thinks it’s not right, it generally isn’t.” However, in about 10% of the instances in which AlphaFold2 was “very confident” about its prediction (a score of at least 90 out of 100 on the confidence scale), it shouldn’t have been, he reported: The predictions didn’t match what was seen experimentally.

> That the AI system seems to have some self-skepticism may inspire an overreliance on its conclusions. Most biologists see AlphaFold2 for what it is: a prediction tool. But others are taking it too far. Some cell biologists and biochemists who used to work with structural biologists have replaced them with AlphaFold2 — and take its predictions as truth. Sometimes scientists publish papers featuring protein structures that, to any structural biologist, are obviously incorrect, Perrakis said. “And they say: ‘Well, that’s the AlphaFold structure.’” ...

> Jones has heard of scientists struggling to get funding to determine structures computationally. “The general perception is that DeepMind did it, you know, and why are you still doing it?” Jones said. But that work is still necessary, he argues, because AlphaFold2 is fallible.

> “There are very large gaps,” Jones said. “There are things that it can’t do quite clearly.”

2 comments

> However, in about 10% of the instances in which AlphaFold2 was “very confident” about its prediction (a score of at least 90 out of 100 on the confidence scale)

I wonder what that confidence score means... If it is 90% probability, then we'd expect it to be wrong 10% of the time

If you're actually interesting, you can read about the scoring here:

https://www.ebi.ac.uk/training/online/courses/alphafold/inpu...

Long short, it's a lot more complex than just % probability the atoms have proper Cartesian coordinates.

Tale as old as ML: people don’t understand it’s an assistance tool, and instead assume it’s always right.

Crazy how we tend to wave away even errors rates of 1% or less. One in a thousand is a lot.

It really depends on the context; (Some) LLMs look impressive specifically because the error rate is comparable to a high score on an exam… the mistake is that even if it was a straight-A student (it's too alien to be that) then it would still only be a student, and we don't put fresh graduates in charge of everything important.

I don't have the domain knowledge to even guess how good 90% is in molecular biology research.

> we don't put fresh graduates in charge of everything important.

We put dribbling halfwit morons in charge of everything. I'm thinking of Liz Truss as UK Prime Minister, but I'm sure most countries have their own examples.

"Anything" != "Everything", and examples like the Iceberg Lady are usually followed with "and that's why they went bankrupt, so don't do that".
This tool is designed to be helpful right now. Looking ahead, there's no reason why AI can't eventually match, or even surpass, human intelligence across the board.

Whether it's advancements in LLMs, with features like long-term memory, or breakthroughs in other areas of ml, it's not guaranteed that humans will remain needed in the research process.

> Looking ahead, there's no reason why AI can't eventually match, or even surpass, human

> intelligence across the board.

There is a reason, actually: what is presently called an "AI" has no concern for the truth. It is a bullshit machine that aims to mimic the right answer.