Also if it's really the date it's nowhere near 32 bits of entropy. I'm guessing you can pretty easily guess to the day when a Bitcoin wallet was created, so that's about 16 bits of entropy. Less if you know the time, possibly 0.
It actually uses the most precise 32 bits of the date, so it's any, like, nanosecond between 0 and some other small amount of seconds. You can't brute force a wallet by knowing approximately when it was made, but you can brute force every mnemonic if you have the time or a bit of cash to throw at a server.
EDIT: It loops around to 0 every 4.something seconds, so it's not like everything after 4 is the same key. It's just a more random distribution than what you may be thinking.
There is often very low entropy in the lowest few bits of system time as well (due to the underlying clock having a different resolution than the system call). Given that every bit you lose halves the time for a brute-force, that's a problem.