Aside from the fact that we'd notice something that radioactive today. It is easy enough to hide something from being seen, it would be harder to hide it (especially something as carefully monitored as radiation) from a technologically advanced civilization.
https://en.wikipedia.org/wiki/Natural_nuclear_fission_reacto...