I don't think physical integration within one contained is relevant to system level behavior.
[1] https://en.wikipedia.org/wiki/Neuroanatomy_of_memory
Your own long term memory is the orchestration of systems that make it long term.
Now, my opinion is it currently can't think, but it certainly has memory.
However, LLMs don't have memory. That's what I (and others on this thread) responded to, which is unrelated to how my own memory works.