[rollcat]

The characters in "Star Trek" are well aware of the holodeck's limitations. Data can "see" the walls:

https://youtu.be/uWWn0fPbcRs?feature=shared&t=192

B'elanna Torres (and many other characters, mostly Klingon) are notorious for disabling the holodeck safety protocols, because the experience is not "real enough" unless one can actually get hurt:

https://www.youtube.com/watch?v=30TXMZ_TSjk

The awareness of the holodeck experience being fake is so pervasive, that Odo actually believes the real Kira Nerys is a holodeck character:

https://youtu.be/8_nfZGFEm5k?feature=shared&t=82

I think this is the correct direction for AR/XR development (and why I'm a fan of AR as a concept, but not necessarily of VR): rather than trying to imitate a real experience as closely as possible (and then inevitably nose-diving into the uncanny valley), work with the medium and its inherent limitations and just let it be its own thing. We might discover applications we wouldn't dream of, just like the EMH from Voyager eventually rediscovers himself as a real person.