You can use a hash of the site appended with a .pass wide pepper as the name of the directory storing credentials for a particular site, then use a wrapper script that hashes its input before passing it to pass.
This is all a lot of effort, if I went down that road I might as well skip "pass" and handle the passwords myself. What I like about pass is that there isn't much setup.
Full disk encryption also doesn't prevent a running application from seeing the directory structure. But I guess this is not a very realistic attack vector.
freepass [0] seems like it could be a potential candidate, somewhere between pass and LessPass, but I haven't tried it out for myself yet.
[0]: https://github.com/myfreeweb/freepass