[scott_karana]

Does the Go http library asynchronously serve static files, by default? Or is it going to block on any app requests?

Nginx is extremely fast for that case, which is typically the reason most people proxy languages through it. ;-)

[vertex-four]

Go I/O is async by default, I figured, so...

It could be that nginx is more efficient at static file serving, but that'd be down to being specifically designed and optimised for it rather than some "sync vs async" thing.

[ekimekim]

Minor quibble, in the context of serving static files (ie. from disk), go doesn't use async I/O, the file I/O blocks the thread until it's complete. But since go's scheduler is M:N this doesn't lock up the whole program, so your point stands.

[vertex-four]

Err, no, this is a misconception. All IO in Go is async - there is no sync IO in Go (as sync IO would block an entire OS thread). There is an internal registry mapping blocked file descriptors to goroutines - when a kernel IO function returns EAGAIN, the goroutine throws the file descriptor + goroutine info onto the registry and yields back to the scheduler. The scheduler occasionally checks all descriptors on the registry to mark goroutines that were waiting on IO as being alive. The scheduler is, therefore, essentially a multithreaded variation on a standard "event loop" - the only difference is that "callbacks" (continuations of a goroutine) can be run on any of M threads rather than just one.

From a Go programmer's perspective, this looks like "blocking a thread", but because goroutines are relatively lightweight in comparison to actual threads, it behaves similarly resource-wise to callback-based async IO. (Although yes, nginx is likely optimised so that it throws out data earlier than Go can free stack space and so can save some memory. Exactly how much is up to benchmarking to find out.)

Basically, the only differences between Go and e.g. a libev-based application as far as IO is concerned is a different syntax - the event loop is still there, just hidden from the programmer's point of view.

Note that this doesn't mean you shouldn't put nginx in front of Go to serve static files - nginx is likely more optimised for the job than Go's file server, might handle client bugs a little better, is more easily configurable (e.g. you can enable a lightweight file cache in just a few settings), you don't have to mess around with capabilities to get your application listening on port 80 as a non-root user, and so on and so forth.

[ekimekim]

I'm referring specifically to disk IO, which on linux using standard read(2) and write(2) is (almost) always blocking. What you describe is true of socket fds and some other things, but on most systems a file read/write which goes to a real disk will never return EAGAIN.

This is why systems like aio[1] exist, though afaik most systems tend to solve this with a thread pool rather than aio, which can be very complicated to use properly.

[1] http://man7.org/linux/man-pages/man7/aio.7.html

[vertex-four]

Ah, absolutely, I forgot that the state of disk IO on Linux is terrible - although this still isn't quite the case, since there's a network socket involved in copying from disk to socket, so if the socket's buffer becomes full the scheduler will run.

It seems that nginx can use thread pools to offload disk IO, although doesn't unless configured to - by default disk IO will block the worker process. And FreeBSD seems to have a slightly better AIO system it can use, too.

[marktangotango]

I would be surprised if go did not use the 'sendfile' syscall that does exactly this, is there a go nut that can clarify?

[Vendan]

Go does shortcircuit to sendfile where possible.