Hacker News new | ask | show | jobs
by deathanatos 333 days ago
JSON encoding is, as someone else points out, a GIL problem, but I want to add that even if you do JSON encoding in an async context:

  async def foo(…):
    json.dumps(d)  # you're blocking the event loop
You're still going to block on it.

  def sync_foo(…):
    json.dumps(d)  # you're holding the GIL … and so blocking here too
Short of resolving the GIL somehow (either by getting ridding of it, which I think is still a WIP though it has been "merged", I believe) or subinterpreters, etc., JSON is inherently going to need to hold the GIL while it walks the structure it is encoding. (Unlike a large file I/O, where it might be possible to release the GIL during the I/O if we have a strong ref to an immutable buffer.)
1 comments

This is more of a usability problem. In the second example, it's obvious that `json.dumps()` blocks everything else and it can be readily observed. It's not obvious that it blocks in the former and I've encountered many surprised coworkers despite it seeming obvious to me.

I think a lot of people assume you can slap `async` onto the function signature and it will not block anything anymore. I've had PRs come through that literally added `async` to a completely synchronous function with that misunderstanding.