Hear hear. The user interface for my first app took me 3x as long as I expected, despite finding (unexpectedly) that Cocoa Touch to be one of the nicest API's I've encountered. It was because I re-built it several times to make it feel "right", and after watching users try it. Each time it got simpler. It probably only took a few days worth of coding for what went into the final app (the editing interface, not the whole thing) but the road to get there was long. I guess that's the value of an experienced UX designer -- they can do that work beforehand with paper and intuition.
The simpler you make something, the more perfect the execution of each of the remaining elements have to be, because your attention is now laser-focused on them.