[acqq]

The most important Andrei's quote:

"D has resolutely exited C++'s shade because it is good at things C++ is not good at, in addition to being good at things C++ is good at. Also, Go would be tenuous to frame as a better C because it cannot do everything C does - e.g. unsafe memory access and manual memory management, which are needed in certain systems - and it interoperates poorly with C."

In short, he states that D can do unsafe memory access and manual memory management. Go can't.

[kisielk]

Note that the discussion in question dates back to 2010, before the existence of cgo. Go now has quite good C interoperability. And Go can now do unsafe memory access and manual management using the "unsafe" package.

[javajosh]

Yes, if I was to learn a systems programming language, I would only pick from those that allow manual memory manipulation. Either that or the ability to trivially integrate with C functions that can do manual memory manipulation.

[sambeau]

I fear you are confusing systems programming language and operating systems programming language (or conflating).

Go is a great systems programming language especially for modern concurrent systems — http servers etc. It talks to C easily so you can integrate all kinds of system level code into your apps. Having a Garbage collector makes it trivial to write long-running daemons.

Modern operating systems have traditionally been written in C (and C++) plus some assembly language code. D can do everything C & C++ can do but it would still no doubt need the assembly language code.

Go lacks manual memory management. Some say that this would be a barrier for writing an operating system while others don't. The fact that you would have to use some assembly language code to talk to the hardware and you might need to add some manual memory management via assembly language code. After that I'm sure the garbage collector would make the OS more reliable and potentially a little quicker in places.

Either way I don't see why Go gets criticised for needing a bootstrap layer when operating systems written in C and C++ also need this.

[bad_user]

There are tons of user-level applications that need manual memory management and that don't have anything to do with operating systems, like for instance video encoding/decoding/streaming or high-frequency trading.

What really bothers me about Go is not really that it's garbage collected, but rather that its garbage collector sucks so badly. Really, not all garbage collectors are created equal.

For example I'm working on a startup and we've been integrating with various bidding exchanges for serving targeted ads. All the bidding exchanges want the response to be generated in under 100ms, which includes the network roundtrip. This means on the server-side, the average must not be higher than 10ms per request, preferably lower. Scala on the JVM can handle it, but when I tried out Go, it was a disaster ... as that garbage collector stops the world and it's totally unpredictable, so you end up with spikes of latency that can upset your partners and given enough incoming requests, it can also blow up your buffers/queues, crashing your servers. It's also non-compacting, but that's a given, as it's not even fully precise yet.

Which is the reason for why integrations with bidding exchanges are usually written in C++ too (in case it's not clear, we are talking about B2B web services). We've gone with the JVM because it provides a good productivity/performance balance, but eternal vigilance is needed in profiling the memory allocation patterns and tuning the garbage collector to handle the load. And Go requires even more tuning. Which is why sometimes I fantasise about a high-level language that allows for manual memory management, as things would be so much easier ... although I'm rooting more for Mozilla's Rust, than I am for D.

[mnembrini]

Sociomantic is a German based company that does online real-time bidding and is using D to do it: you can find more info in their blogpost about Dconf 2013 with links to two talks talking about their company and how they dealt with the GC issues you mention.

https://www.sociomantic.com/blog/2013/06/dconf-2013-review/

[simplekoala]

Interesting you chose Scala. I have built and managed Jetty based adservers to talk to bidding exchanges and wondered how would the system perform if the adserver was written in Scala instead. Going with Jetty initially was the right choice as none on the team were experts in Scala, and the base assumption was that Jetty is battle tested, has good documentation and is already optimized to perform great from the get go. So, we simply chose Jetty. I didn't get a chance to build an equivalent system in Scala to compare performance. Will be great, if you can throw some light on how you went about building highly performant adservers in Scala, and scaled them. Does coding in Scala make it a lot more easy to deal with concurrency (for development and debugging), keeping a small codebase, help in rapid iteration, and in general make it fun to program and manage the servers? Do you have a blog post or a write up on this topic? I encourage you to make one, if you don't have one already. Good karma, and an opportunity to showcase your engineering chops to hire other great engineers.

[fedesilva]

I wrote a simulation software to test a product in my previous job which used akka mostly but also used AHC which is a plain java http client library without problems. Size-wise the code would periodically get more features but by refactoring often it never got beyond 2500~3000 LOCs. Readability was never harmed; Scala - contrary to what some people think - allows for very clear code. People that inherited the code is working on adding features. Obfuscated code can be written in PHP if you want (or are unable to do otherwise). Another piece of scala software was an internal web service written using unfiltered on jetty. It's so stable sometimes I just forget it's there.

[zoul]

I think that Objective-C would qualify as a reasonably high-level language and offers a choice between manual and automatic, but very predictable memory management.

[sjwright]

Plus, it's also pure C whenever you want.

[mratzloff]

I'm building a similar infrastructure (different purpose). Can you give a few more details about the problems you ran into?

What version of Go?

Message rate per second?

Message size?

What was the % increase over average response time during GC?

What class of hardware?

I did notice the Go's GC improved between 1.0 and 1.1.

Thanks!

[pjmlp]

> Modern operating systems have traditionally been written in C (and C++) plus some assembly language code. D can do everything C & C++ can do but it would still no doubt need the assembly language code.

Actually, contrary to C and C++ language specification, in D, support for inline assembly is part of the language specification.

> downvote

sambeau 50 minutes ago | link | parent | flag

I fear you are confusing systems programming language and operating systems programming language (or conflating).

Go is a great systems programming language especially for modern concurrent systems — http servers etc. It talks to C easily so you can integrate all kinds of system level code into your apps. Having a Garbage collector makes it trivial to write long-running daemons.

Modern operating systems have traditionally been written in C (and C++) plus some assembly language code. D can do everything C & C++ can do but it would still no doubt need the assembly language code.

Go lacks manual memory management. Some say that this would be a barrier for writing an operating system while others don't. The fact that you would have to use some assembly language code to talk to the hardware and you might need to add some manual memory management via assembly language code. After that I'm sure the garbage collector would make the OS more reliable and potentially a little quicker in places.

While I am on the D and Rust field, I support Go's ability to do this.

Go is no different from Oberon in system capabilities. And Oberon was used to write quite a few desktop systems used at Zurich's ETHZ during the mid to late 90's.

The OS bootloader and the kernel package for hardware interactions were written in Assembly, with the remaing parts in Oberon.

You can read all about it here, http://www.inf.ethz.ch/personal/wirth/books/ProjectOberon.pd...

> Either way I don't see why Go gets criticised for needing a bootstrap layer when operating systems written in C and C++ also need this.

Because many tend to assume compiler extensions to C and C++ are part of the language.

It is a fun quiz asking C and C++ developers what is part of the standard and what is compiler specific behaviour.

[fauigerzigerk]

>I fear you are confusing systems programming language and operating systems programming language (or conflating).

Systems programming doesn't seem to be a very well defined term. My understanding is that it is certainly not application programming and it requires pretty tight management of hardware resources. That includes things like operating systems, database systems, embedded systems, networking software like firewalls, etc.

[andrewflnr]

The way I like to think of it is that a "systems langauge" is something you could use to implement a "real"[0] GC. Does this ring true to anyone else?

[0] You know what I mean.

[pjmlp]

A systems language is one with enough features to allow developers to write kernel code and device drivers, even if with a little help from assembly.

A GC is just a special case of a memory manager.

[agumonkey]

my own take :

systems language => for building systems, many parts, many interconnections

operating system language => ability to use the hardware with no limitation. even for a monolithic part.

[he_the_great]

> systems language => for building systems, many parts, many interconnections

So... every language?

[mwfunk]

I guess it's a difference in terminology. I've never heard anyone refer to writing an HTTP server as "systems programming"; I'm used to "systems programming" meaning the same thing as "operating systems programming". Sometimes people refer to writing higher level OS components (like init or libc) as "systems programming" as well, but I would think that a server that isn't a core part of an OS would just fall under the category of "server programming".

[duaneb]

I would argue go has a terrible ffi due to their tool chain being awkwardly integrated (at best) with the native c abi. Calling into c is expensive.

[howeman]

Go can do unsafe memory access, can't it? See http://golang.org/pkg/unsafe/

[georgemcbay]

Go can do unsafe memory access, but doing things like pointer arithmetic or casting of a memory blob to an arbitrary struct type is way more painful than it is in C/C++.

In a lot of contexts, this is a feature (makes terrible code smells in normal code easier to see), but depending upon the type of coding you do, you do sometimes find yourself wishing it were a bit easier when dealing with things like graphics APIs where you just want to lock a texture, update some bits in-place, and then unlock it. Sometimes what might be a couple lines of C code are 10s of lines of Go code where you either do crazy gymnastics with the unsafe package, or juggle things in and out of byte buffers.

[derefr]

Sounds like D would be better-compared to Rust, then.

[pcwalton]

Rust has safe manual memory management, at the cost of a learning curve. In D you must use the garbage collector if you want memory safety, but it avoids all the complexity of lifetimes and uniqueness. This difference makes the two languages feel pretty different, even if at their core they're pretty similar.

[luikore]

D is more like C++ without macro and with GC plus some engineering features (unit test is a language feature? weird). Rust looks different and does different. For example, idiomatic foreach block in Rust is a syntax sugar of passing lambda, which is more friendly to parallelism. While in C++/D it is a syntax sugar of classic for-loop with iterator, which is more efficient in unparalleled environment.

[dbaupp]

> For example, idiomatic foreach block in Rust is a syntax sugar of passing lambda, which is more friendly to parallelism

This changed with the recent 0.8, `for` is now syntax sugar for using the Iterator trait, which optimises exactly like C++ (the vector iterator even vectorises when LLVM can do it).

Rust has actually moved to have iterators that are very similar to (a subset of) D's ranges.

[WalterBright]

You might thing that unit test as a language feature is weird. But our experience with it is that minor-seeming weird feature has been an enormous success.

It has literally transformed writing D code, and for the better. There's another little feature, -cov, which will tell you which lines of code are covered by the unit tests.

It's hard to understate the improvement these engender.

[_ak]

Go can do unsafe memory access. Andrei has no idea what he's talking about.

[andralex]

I actually think I do. Go does offer a couple of primitives for poking into memory, but it is my opinion that a systems language needs much more refined control over memory layout and allocation.

[acqq]

_Ak, meet Andrei:

http://dotsub.com/view/892ed71a-0d53-499d-aaa8-d5e642f2942c

Still, I think you raised valid question in the first part of your message, though the second is a bit offensive and I also thank Andrei for his clarification and use the opportunity to congratulate for this milestone of D language. I like D and even if I use C++ at work, I know I'd enjoy using D especially for the features that let my code "see" and "build" the other parts of the code.

[jmdavisProg]

I think that a lot of confusion has been caused by Go using the term "systems language," because (as I understand it) Go doesn't mean it in the same way that C++ and D do. The Go folks seem to be thinking systems in the sense of large networks of computers and the like (the kind of stuff that Google typically does), whereas the C++ and D folks are thinking systems in terms of stuff like operating systems. What Go is trying to do does not necessarily require low-level primitives (though it can benefit from them), whereas what C++ and D are trying to do does require such primitives.

[he_the_great]

The problem is by deciding that they can use "systems language" (which they dropped) because "system" means "a set of connected things or parts forming a complex whole" results in every language falling under "systems language."

Others have been using it to distinguish languages suitable for writing operating systems/drivers years before Go introduced this confusion.

[jdhzzz]

When I hear "systems programmer" I think computer-to-computer. Therefore when I hear "systems language" I think computer-to-computer even absent Go's use of the term. I not be disappointed if a systems language were not appropriate for creating operating systems.

[_ak]

Now does it allow unsafe memory access, or does it not?

And does the unsafe package allow you to build a C-style manual allocator or not (regardless of whether it integrates with Go's new or make operators)?

[luikore]

If Go can do, most languages can do, in an easier way.