[titzer]

AIs should be forced to show their work. Every tool they use, every program they generate and run in the background, and every logical inference. They should be forced to produce Lean or Rocq proofs or execution traces for all the computation they use. For facts, they should be able to produce sources. For any abstract reasoning, they should be able to break it down into explainable steps.

Then, on top of that, they should be able to explain any of that, at any level of detail, whether talking to an expert or a layperson.

[chickensong]

That's an implementation preference, and you can have all of that today if you write your own harness. Vibe coding a cat meme generator doesn't need a Lean proof.

[TacticalCoder]

> AIs should be forced to show their work. Every tool they use, every program they generate and run in the background, and every logical inference.

And they should be 100% deterministic: same input (including a random seed / "temperature" if one wants), same output.

On a not totally dissimilar note I like the projects saying: "AI PRs aren't accepted but if you do something worthwhile, give us your prompts instead".

[jstanley]

Conversely, https://www.lesswrong.com/posts/fzeoYhKoYPR3tDYFT/beware-iso...

[XorNot]

AIs aren't people. I can demand whatever I want from them.

The entire premise of building machines that can provide intelligence is that they can be arbitraryly rigorous and do far more work then is reasonable or practical to demand of a human.

[hackernewds]

why?

[red75prime]

How else would you trust a tangled mess of neurons to produce something that works? Artificial neurons, of course, hehe.