Evolution does not optimize for anything. If the organism propagates, it may propagate again. But there is no goal.
As the RQ shows, this process often leads to a dead end. Such as a short term success for cancer, but no long term success. Deadly infections lose their deadliness over time, as killing the host does not lead to propagation.
Evolution often falls into a local optima, which will inevitably lead to extinction.
Deadly diseases losing their deadliness over time is possible, but hardly guaranteed even at the species/population level. Rabies has effectively a 100% fatality rate in host species. Smallpox, which is human-specific with no animal reservoir so must have been spread consistently and entirely within humans, had a fatality rate on the order of 10-30% even after thousands of years of co-evolution.
Smallpox was much less deadly to Europeans than the Indians. Indians fell like flies to European diseases.
Covid seems to have its mortality dramatically shrunk.
Our genomes are full of bits and pieces of ancient disease DNA. Our bodies are full of bugs that have evolved into peaceful coexistence. Some bugs even became part of us (mitochondria).
> Smallpox was much less deadly to Europeans than the Indians. Indians fell like flies to European diseases.
That does not support your argument that there is a adaptive advantage for reduced deadliness. The fact that it was exceedingly deadly to non co-evolved hosts indicates it was not the disease that became less deadly, but that the co-evolved hosts developed better defenses.
> Our bodies are full of bugs that have evolved into peaceful coexistence.
That is a argument that there is a continuous adaptive advantage to reduced deadliness down to ~0%. Again, Rabies had and continues to have a nearly 100% fatality rate in co-evolved hosts for thousands of years. Smallpox had a 10-30% fatality rate. Any magical inherent adaptive advantage for reduced deadliness failed to materialize to continue pushing down their deadliness.
Or put another way, a disease can have a 30% mortality rate and still do a really bang-up job at propagation with limited adaptive pressure to reduce that further for thousands of years. Peaceful coexistence is more likely a artifact of the specific dynamic than any sort of meaningful fundamental advantage to reduced deadliness.
> it was exceedingly deadly to non co-evolved hosts indicates it was not the disease that became less deadly, but that the co-evolved hosts developed better defenses
That's a good argument, but it is not proof that there wasn't some adaptation of smallpox to Europeans. The immune systems of Europeans and Indians diverged 10,000 years ago.
> Rabies had and continues to have a nearly 100% fatality rate in co-evolved hosts for thousands of years.
I doubt that there were large enough epidemics of rabies to influence its evolution.
> Peaceful coexistence is more likely a artifact of the specific dynamic than any sort of meaningful fundamental advantage to reduced deadliness.
Killing your host does not help propagation of the disease. Causing your host to cough and sneeze is a great way to propagate.
Not saying that evolution has a will, but the mechanism is that those species that are best adopted will prosper and go one. So that species that can reproduce the best, wins. So I don't see why you disagree that evolution does not optimize for it. (No one said anything about perfectly optimized)
As the RQ shows, this process often leads to a dead end. Such as a short term success for cancer, but no long term success. Deadly infections lose their deadliness over time, as killing the host does not lead to propagation.
Evolution often falls into a local optima, which will inevitably lead to extinction.