Well, Germans without a passport can still travel all around the Europe. Passport is only needed if you plan to go outside EU - that should be a factor in this comparison.
An European citizen can go to Egypt or Estonia or the Canary Islands or Iceland or Norway or Switzerland without a passport
OTOH it's easier to learn English for an European - we all study it in primary schools - and go to the US than for an American to learn Norwegian and move to Norway.
It's also a lot less of a cultural shock to go EU->USA than the other way around.