[bad_user]

There are tons of user-level applications that need manual memory management and that don't have anything to do with operating systems, like for instance video encoding/decoding/streaming or high-frequency trading.

What really bothers me about Go is not really that it's garbage collected, but rather that its garbage collector sucks so badly. Really, not all garbage collectors are created equal.

For example I'm working on a startup and we've been integrating with various bidding exchanges for serving targeted ads. All the bidding exchanges want the response to be generated in under 100ms, which includes the network roundtrip. This means on the server-side, the average must not be higher than 10ms per request, preferably lower. Scala on the JVM can handle it, but when I tried out Go, it was a disaster ... as that garbage collector stops the world and it's totally unpredictable, so you end up with spikes of latency that can upset your partners and given enough incoming requests, it can also blow up your buffers/queues, crashing your servers. It's also non-compacting, but that's a given, as it's not even fully precise yet.

Which is the reason for why integrations with bidding exchanges are usually written in C++ too (in case it's not clear, we are talking about B2B web services). We've gone with the JVM because it provides a good productivity/performance balance, but eternal vigilance is needed in profiling the memory allocation patterns and tuning the garbage collector to handle the load. And Go requires even more tuning. Which is why sometimes I fantasise about a high-level language that allows for manual memory management, as things would be so much easier ... although I'm rooting more for Mozilla's Rust, than I am for D.

[mnembrini]

Sociomantic is a German based company that does online real-time bidding and is using D to do it: you can find more info in their blogpost about Dconf 2013 with links to two talks talking about their company and how they dealt with the GC issues you mention.

https://www.sociomantic.com/blog/2013/06/dconf-2013-review/

[simplekoala]

Interesting you chose Scala. I have built and managed Jetty based adservers to talk to bidding exchanges and wondered how would the system perform if the adserver was written in Scala instead. Going with Jetty initially was the right choice as none on the team were experts in Scala, and the base assumption was that Jetty is battle tested, has good documentation and is already optimized to perform great from the get go. So, we simply chose Jetty. I didn't get a chance to build an equivalent system in Scala to compare performance. Will be great, if you can throw some light on how you went about building highly performant adservers in Scala, and scaled them. Does coding in Scala make it a lot more easy to deal with concurrency (for development and debugging), keeping a small codebase, help in rapid iteration, and in general make it fun to program and manage the servers? Do you have a blog post or a write up on this topic? I encourage you to make one, if you don't have one already. Good karma, and an opportunity to showcase your engineering chops to hire other great engineers.

[fedesilva]

I wrote a simulation software to test a product in my previous job which used akka mostly but also used AHC which is a plain java http client library without problems. Size-wise the code would periodically get more features but by refactoring often it never got beyond 2500~3000 LOCs. Readability was never harmed; Scala - contrary to what some people think - allows for very clear code. People that inherited the code is working on adding features. Obfuscated code can be written in PHP if you want (or are unable to do otherwise). Another piece of scala software was an internal web service written using unfiltered on jetty. It's so stable sometimes I just forget it's there.

[zoul]

I think that Objective-C would qualify as a reasonably high-level language and offers a choice between manual and automatic, but very predictable memory management.

[sjwright]

Plus, it's also pure C whenever you want.

[mratzloff]

I'm building a similar infrastructure (different purpose). Can you give a few more details about the problems you ran into?

What version of Go?

Message rate per second?

Message size?

What was the % increase over average response time during GC?

What class of hardware?

I did notice the Go's GC improved between 1.0 and 1.1.

Thanks!