I can't afford not to be a U.S. citizen. I'm a non-millionaire atheist black man with a Muslim name and an American accent. My kind don't last long in the wilderness, I will be traded by intel agencies like a baseball card.
If you aren't a permanent resident you are almost always tied to your current job, and can technically be required to leave within two weeks if you are laid off or fired. (Getting a new job that would accommodate your visa requirements on a two-week notice might not be trivial.) If you are a permanent resident, you are subject to many of the annoyances discussed (i.e., IRS) without the benefits of citizenship...
He was talking about not living in the USA specifically, so your not tied to your job like that, and you don't have any US tax requirements unless the income comes from the USA.
The tax system for overseas income is complicated, especially if you hold foreign assets (bank accounts or corporate interest), but the short version is that any income earned as an employee up to ~$95,000 can be excluded. It's called the foreign earned income exclusion.
Don't forget that if you live in a country that has a tax treaty with the US, any taxes paid to your home gov't counts towards any taxes you would pay to the US.
So if you live in a higher tax jurisdiction (i.e. the majority of the countries in the world), your US tax bill ends up being $0. However, you still have the hassle of filing your US tax return each year.
I didn't think it was that complicated at all. I held assets in Canada while working in the US and all I had to do was declare what they were and how much.
To add to this, you legally have to declare to the U.S. government all foreign cash in excess of $10k. You may not pay tax but you definitely have to file regardless.
As a U.S. citizen you are required to pay taxes on income earned worldwide and from my understanding you can only deduct up to ~95k in 2012.
From what I've read, being a US citizen and working abroad is expensive and one of the main motivations for people to renounce their US citizenship.