Does anyone know how good is Lua's support for async IO? Especially the handling of large amount of connections and the memory footprint for each connection?
Luasocket does async IO. It uses poll or select on Unix (calling it 'select'); I'm not familiar with what it uses on Windows. It's not "web scale", but is very easy to use, and works well enough for a couple hundred simultaneous connections.
Lua's convenient C API means that wrapping libev, libuv, or libevent is really not hard, if you want to go that route. (I have a libev wrapper on github, FWIW.)
The thing to keep in mind is that if you're writing high-throughput servers, you probably want to avoid parsing the network IO via Lua. While Lua strings are fine with arbitrary binary data (i.e., \0s are fine), it interns all strings; you'll have the overhead of converting every read into an atom/symbol, and it will spend a lot of time garbage collecting. (Lua makes this daring trade-off because it's usually a net win, and when it isn't, you still have the option of doing things in C instead.) Lua's great for handling all the control logic, though. Coroutines are particularly applicable there.
In a nutshell: If you're good with C, and understand the issues involved with performant async IO, Lua can work very well.
Thanks for the info. Very informative reply. Sounds like the network handling stuff should be in the C layer. And pass the parsed message/objects back to Lua is the way to go.
I'll spare you the exact results of my recent web search, but my impression is that it seems to be a rapidly developing area. There are definitely some good libraries out there of varying scope and maturity.
Lua seems to be such a good fit for this paradigm that I would bet the community is going to produce some outstanding frameworks. (Was kicking around the idea of implementing something myself.)
Lua's convenient C API means that wrapping libev, libuv, or libevent is really not hard, if you want to go that route. (I have a libev wrapper on github, FWIW.)
The thing to keep in mind is that if you're writing high-throughput servers, you probably want to avoid parsing the network IO via Lua. While Lua strings are fine with arbitrary binary data (i.e., \0s are fine), it interns all strings; you'll have the overhead of converting every read into an atom/symbol, and it will spend a lot of time garbage collecting. (Lua makes this daring trade-off because it's usually a net win, and when it isn't, you still have the option of doing things in C instead.) Lua's great for handling all the control logic, though. Coroutines are particularly applicable there.
In a nutshell: If you're good with C, and understand the issues involved with performant async IO, Lua can work very well.