Even then 90% of the 1040-EZ is instructions on how to actually use the form. If you cut all that out and just have the boxes to fill with labels it's not much bigger than the post card it's just needlessly hard to file.
I recall seeing recently (somewhere in Europe, I think) the idea that because the government tracks this information anyway as people get paid throughout the year, all that is required is basically you (the taxpayer) attesting that the information being submitted is correct.
At that point, you just sign the form and send it back.
Yeah, the only place it really breaks down is where you're on the line between the standard deduction and any itemized deductions and finding out requires going through the whole process with itemized to see if you come out better. I've been right on that line since buying a house and it's annoying every year.
It's also unlikely to happen because a dedicated faction in the GOP specifically want taxes to be as annoying as possible to make people resent it more.
At that point, you just sign the form and send it back.