That's basically what ends up happening. Over 40% of people don't respond to the initial audit notice, so what ends up happening is the adjustment is defaulted in the government's favor. See https://www.taxpayeradvocate.irs.gov/news/nta-blog-eitc-audi...