Does the distinction between sharing VA mappings and copying buffers to/from kernel matter from a security perspective? (I assume it does, but I don't know why.)
Yes, you're looking at kernel pages through userspace virtual memory mappings, this isn't the case with copy to user. You're just copying data from a userspace page to a kernel page, but only in kernel mode. You don't get to "see" kernel pages and in fact post spectre/meltdown the kernel is unmapped in userspace.