How does that work? I'm not familiar with Ray, but I'm assuming you might be referring to actors [1]? Isn't that basically the same idea as multiprocessing's Managers [2], which also allow client processes to manipulate a remote object through message-passing? (See also DCOM.)
According to the docs, those shared memory objects have significant limitations: they are immutable and only support numpy arrays (or must be deserialized).
https://docs.ray.io/en/latest/ray-core/objects.html