Pompeii was literally buried in a matter of hours, so that’s kind of a unique case. Any underground structures would be turned into a cave from water seeping in. Most people don’t dig into the side of caves, so it’s likely we’d never know. A great example is the millennium old water passages in Afghanistan. You can’t tell they are man made any more.
The point stands, most things don’t survive all that long (in the grand scheme of things). Occasionally, the conditions are exactly right to preserve something, but there is a limit.
Pompeii looked remarkably fresh when dug out.