[ronaldx]

> I would have thought nature would stop optimizing once a sting is a good enough deterrent.

I believe you are probably right about this: but the spider venom was probably developed in an arms race with some {predator/prey} species. In your 'tarantula hawk' example, the venom must work on the tarantula, and the tarantula must do its evolutionary best to protect itself from the venom. This combination allows the venom to become extremely effective.

Or, given that it's the black "widow", maybe the venom is a result of sexual selection[0], which tends to have crazily extreme results. A spider fetish, if you like.

But, the sting/venom is not designed for humans (it just happens to work on them), and humans as a species have no reason to develop better protection. Humans typically aren't a target of the bites and almost always recover with no long-term effect. It isn't really designed to work on humans. But, it still hurts (and maybe that's a nice side benefit for a reclusive spider).

[0] https://en.wikipedia.org/wiki/Sexual_selection