[claytonius]

Something like a CycleGAN and/ or weight sharing would do the trick