That's probably the case for you and I, but I'm sure with sufficient knowledge of how DFU works (for example by employing the engineers who designed it) you can persuade it to only rewrite particular blocks, leaving the data intact.
The key is not the password. If that were the case, the phone would have to re-encrypt everything everytime you change the pin or password. The password unlocks the key. And if you brute force the key itself, it might take decades, maybe centuries.
You could rewrite the bootloader and recovery firmware to really do anything. Resize some partitions and dual boot into a shell with some drivers for keyboard and wireless.