Unless something has changed in the past few years, you kind-of-sort-of-maybe need to do that. The US doesn't actually enforce anything like it, but all of my naturalized coworkers were required to surrender their previous passport when they took the oath of US citizenship. The US gives the passport back to the home country. Canada mails it back to you and says you are still Canadian too, while China does not and says you have to get a visa like any other tourist if you ever want to step foot in China again.
I was naturalized about a decade ago and nothing of the sort happened for me either, or for the other two people I know who were naturalized around that time.
The reason is that you are only required to renounce "allegiance" to other countries during the naturalization ceremony. This has nothing to do with citizenship.
Has this changed?