It was started in 1861 as a way to raise revenue to fight a war.
https://nomadcapitalist.com/global-citizen/citizenship-based...
Deserters would most likely have gone home, remaining citizens and living in the US. And I'm unaware of US IRS attempt to collect taxes from the CSA during the war.
https://teachinghistory.org/history-content/ask-a-historian/...