You don’t need to share objects. Have explicitly shared buffers instead. Python is a dynamic language so you can easily build proxy objects that are views into a shared buffer, and this allows you to keep all your single threaded performance because no objects are shared.
For example:
buf = sharedbytes.alloc(1 * GiB)
with buf.lock(lockid):
buf[10:20] = 42
message_other_process(buf.id)
# other process
def recv_buffer(bufid):
buf = sharedbytes.get(bufid)
with buf.lock(lockid):
print(buf[10:20])
Most people don’t even need that and would be satisfied with just virtual processes and copying message passing between them.
You don’t need to share objects. Have explicitly shared buffers instead. Python is a dynamic language so you can easily build proxy objects that are views into a shared buffer, and this allows you to keep all your single threaded performance because no objects are shared.
For example:
Most people don’t even need that and would be satisfied with just virtual processes and copying message passing between them.