Hacker News new | ask | show | jobs
by albertzeyer 775 days ago
> reference counting is not intrusive in CPython (meaning the reference counting structures are outside the PyObject, last I checked)

That's wrong. That was never the case.

Recent CPython: https://github.com/python/cpython/blob/6d419db10c84cacbb3862...

CPython 2.0: https://github.com/python/cpython/blob/2a9b0a93091b9ef7350a9...

CPython 0.9.8: https://github.com/python/cpython/blob/dd104400dc551dd4098f3...

Regarding multiprocessing.Pool, that would not work as I said. I was thinking more about a plain fork, like this:

    def parent():
        juicy_variable = ...

        def workerfunc(x):
            # I can access juicy_variable
            ...

        childs = []
        for i in [1, 2, 3]:
            child = fork()
            if child == 0:
                workerfunc(i)
                sys.exit()
            childs.append(child)

        # Wait for and cleanup childs.
        # Communicate somehow with childs to get back results.
        ...
1 comments

>> reference counting is not intrusive in CPython (meaning the reference counting structures are outside the PyObject, last I checked)

> That's wrong. That was never the case.

You are right, I was mistaken.

The point stands though, PyObjects are not really an issue for use cases where these tricks are needed.

> I was thinking more about a plain fork, like this

Right well you can recreate a multiprocessing pool of your own with different pros and cons, sure, that's an other approach I guess.