I think so too, but then, (many/most) papers will need to be 2x or 3x longer - which is fine by me, but conferences and journals have strict length limits.
Maybe the scratch notes could be published outside of the paper itself. It’s not an ideal solution, of course, because links can go stale. But maybe it’s a starting point.