Unless I've misunderstood you, if you're dealing with pod IPs instead of service DNS addresses you're not fault tolerant and you might as well not even bother with K8s.
Well you're not necessarily dealing with a single pod IP. Two versions of skipping cluster IPs are either having smart client libraries that are capable of multiplexing between different IPs, or using something like consul DNS to resolve a URL into multiple pod IPs and rely on client DNS to do the right thing.
That is untrue. Client-side load balancing is a common thing. The service construct also doesn't necessarily map to what external clients need. Kafka or Cassandra, for example, prefer client direct access to pods via headless services.