Note that there aren't a lot of ecosystems where everyone follows semver rules. Even on the npm registry, patch version bumps can still very much break your code because there's no validation during publication. It's still on you to make sure you have tests set up that run when code or dependencies change, even for something as simple as a patch version bump.
Although specifically to your example of 0.8.0 to 0.8.1: that's exactly the kind of version that semver guarantees is not safe: major version 0 is the "unstable" version, and the minor/patch rules do not apply to it (see https://semver.org/#spec-item-4).
Although specifically to your example of 0.8.0 to 0.8.1: that's exactly the kind of version that semver guarantees is not safe: major version 0 is the "unstable" version, and the minor/patch rules do not apply to it (see https://semver.org/#spec-item-4).