Even if it doesn’t do that, fixing up various addresses can be a lot of work if the linker can make code shorter (e.g. by using short branches where possible), or if it tries to increase cache locality by changing function order.