Hacker News new | ask | show | jobs
by simonw 2466 days ago
"One; Python has its uses, but writing highly performant multi-threaded scanners is not one of them"

Is that still true given Python 3 asyncio? My understanding is that it's really well suited to writing things like network scanners, without needing to run them in multiple threads.

4 comments

It hasn't been a problem for most of our things at Shodan. I think there's a lot of confusion about when to use multithreading vs non-blocking sockets. Most security tools tend to use multithreading when they should just use non-blocking sockets. With Python3 you can also swap out the scheduler if you want better performance:

https://magic.io/blog/uvloop-blazing-fast-python-networking/

asyncio is still bound by the GIL. It's not like Go's continuations which can parallelize on the CPU. There still can only be one TCP payload from your Python process at one time. The author wanted to spam the network and use all his bandwidth. Python multiprocessing would have worked for this though (probably faster than the solution using shell commands, but obv this is network-bound much more than CPU bound so who cares).
asyncio makes it a lot better, but I still would reach for a concurrency-oriented language such as Golang or a beam language (elixir, Erlang, etc).
depends on your definition of 'highly performant'

While asyncIO hugely helps, this + interpreted language wont yield better performance vs custom native code doing true native threads which themselves are also using async methods.