I was thinking a classical linker that e.g. combined sections together after already laying out subsequent sections. That would basically be an memmove. It doesn't necessarily have to be unsafe, but I could see it being implemented that way just to keep things simpler on the rest of the data structures. (Though if you said that that was an indication of incomplete design I'd have a lot of time for that argument.)
It wouldn't be the linker that has to be unsafe, it'd be the "and now execute!" jump. And that could be abstracted as memfd+execveat, which are fairly normal operations.
OP's argument is roughly "doings things with computers has to be unsafe to be useful", which is.. uninteresting.