And then erase all intermediate steps and just publish the end result - which no one will understand (including yourself a couple of years later), but that's OK /s
I've lamented this effect of the paper writing process for some time. I think a paper should present the steps taken by the authors to arrive at the new results, not just the new results in isolation. Those steps often provide so much useful insight.
I think so too, but then, (many/most) papers will need to be 2x or 3x longer - which is fine by me, but conferences and journals have strict length limits.
Maybe the scratch notes could be published outside of the paper itself. It’s not an ideal solution, of course, because links can go stale. But maybe it’s a starting point.
Increasingly this evolution is spread across multiple papers, and is not enclosed within a single paper. So you can partially follow the chain of thought.