[toast0]

I've been knee deep into TURN lately, so while it would be a terrible idea, I think you could do something like TURN in 'server mode', where the origins are TURN clients and the proxy is a TURN server.

Then when a packet comes in, you decide which origin to forward it to (by unspecified magic), and then send it on annotated with the source ip and port. Replies from the server would include the desired destination ip and port. TURN can also setup a 'channel' so data packets skip the addressing. The end client facing service ip and port would be implicitly linked to the internal TURN service ip and port.

I think the turn server I've been working with could do this out of the box for a single origin server with specific configuration. With a single origin server, it's not very useful other than proof of concept, of course.

But LVS or some other lower level balancing, where ideally the origins are configured with the service ip and port, and outbound packets don't need munging (DSR) sounds like a much better idea. Any sort of rules based forwarding based on a generic UDP packet seems hopelessly complex because of tracking state on stateless protocols.