Or without paying a fee. Which is currently $2,350.
(Ironically, you have to pay more to renounce US citizenship, then you have to pay in all the various filing fees etc when acquiring citizenship through naturalization.)
As a practical matter, you can evade US taxes if you have no financial contact with the US; the IRS can deem you liable for any amount of taxes and it won't have any effect on you if they're unable to collect them.
(Ironically, you have to pay more to renounce US citizenship, then you have to pay in all the various filing fees etc when acquiring citizenship through naturalization.)