[jmtulloss]

The comments so far seem focused on taking a cheap shot, but as somebody working on using AI to help people with hard, long-term tasks, it's a valuable piece of writing.

- It's short and to the point

- It's actionable in the short term (make sure the tasks per session aren't too difficult) and useful for researchers in the long term

- It's informative on how these models work, informed by some of the best in the business

- It gives us a specific vector to look at, clearly defined ("coherence", or, more fun, "hot mess")

[kernc]

Other actionable insights are:

- Merge amendments up into the initial prompt.

- Evaluate prompts multiple times (ensemble).

[sandos]

Sometimes when I was stressed, I have used several models to verify each others´ work. They usually find problems, too!

This is very useful for things that take time to verify, we have CI stuff that takes 2-3 hours to run and I hate when those fails because of a syntax error.

[xmcqdpt2]

Syntax errors should be caught by type checking / compiling/ linting. That should not take 2-3 hours!

[nth21]

There’s not a useful argument here. The article is using current AI to extrapolate future AI failure modes. If future AI models solve the ‘incoherence’ problem, that leaves bias as a primary source of failure (according to the author these are the only two possible failure modes apparently).

[toroidal_hat]

That doesn't seem like a useful argument either.

If future AI only manages to solve the variance problem, then it will have problems related to bias.

If future AI only manages to solve the bias problem, then it will have problems related to variance.

If problem X is solved, then the system that solved it won't have problem X. That's not very informative without some idea of how likely it is that X can or will be solved, and current AI is a better prior than "something will happen".

[nth22]

> That's not very informative without some idea of how likely it is that X can or will be solved

Exactly, the authors argument would be much better qualified by addressing this assumption.

> current AI is a better prior than "something will happen".

“Current AI” is not a prior, its a static observation.