[kstrauser]

Thing is, all interesting Internet services require 2-way communication. Suppose you had an IPv6 client trying to talk to an IPv4 server. With that scheme, the initial packet could get routed to the server by dropping the first 96 bits of the address and treating the rest like an IPv4 address. But where do you put those other 96 bits so that the server can send its reply packet back to the originating client?

You can use NAT64 which does exactly that. If you're doing that anyway, there's not much advantage to adding all the complexity directly into the protocol.