Hacker News new | ask | show | jobs
by andrewl-hn 658 days ago
I haven't done any gRPC programming, but from what I understand it's a modern re-imagining of XML Web Services: you describe the contract and can generate client and server code based on it for different programming languages.

Early on in my career I worked on several projects that used WS-* "stack", and generally it was a very good experience. Once we had a project that was split between two subcontractor teams and each team worked on their portion of a system (a .NET application and an J2EE server) with API based on a common WSDL spec. The two teams (dozens of engineers on each side) worked independently for about a year, and after that they tried to run the two subsystems together, and it was really cool to see the parts "just click". There were some minor issues (like one side expected UTC timestamps and the other were sending localized time) but they took really little time to fix. The fact that two teams were not really talking to each other, were using different languages and libraries, relied on some manual testing through SoapUI and some mocks, and yet the whole thing even run at first attempt was very, very impressive!

WS-* was heavily criticized at the time: the standards and data formats were convoluted, the tooling beyond .NET and JVM was almost non-existent, Sun and Microsoft were not following the standards in their implementations and cared more about the interop with each other than about being standard-compliant. So, ultimately REST and JSON pushed the whole thing away. But I'm really happy to see people trying to replicate what was great about Web Services without making old mistakes, and I wish everyone involved all the best.

Which brings me to my actual question. Since software development history repeats / rhymes with itself every decade or two I now wonder if XML Web Services were not the first iteration of the formula? Was here another popular technology in 70s, 80s, or 90s that used had people describe an RPC contract and then used it to generate client and server glue code for it?

I know that both COM and CORBA used IDL to describe API, but I don't remember any code generation involved.

2 comments

In the 80s there was RPC/XDR (https://en.wikipedia.org/wiki/Sun_RPC, https://en.wikipedia.org/wiki/External_Data_Representation), which was used pretty heavily in the Unix world. On most Unix systems today, including Linux and macOS, you'll find this suite installed. Try `man rpc` and `man xdr`. XDR is what protocols like NFS are based upon.

A contemporaneous competitor to Sun RPC was DCE/RPC (https://en.wikipedia.org/wiki/DCE/RPC), which I think Microsoft's SMB protocol was based upon, albeit in Microsoft's trademark manner--embrace, extend, extinguish.

None of these require compilers, but you're usually better off for them, especially for serialization and deserialization, regardless of whether you're using a specialized library. On Unix there are libraries that can be used in an ad hoc fashion for RPC/XDR, but also rpcgen (`man rpcgen`).

Sun RPC — the official name was ONC RPC — was probably the first modern RPC. It's an Internet standard. You've probably used it without realizing it; it's the protocol that NFS uses. If you've ever had to deal with the NFS "portmapper", then that's because of Sun RPC. Some other protocols use it.

It uses XDR as the schema definition language. XDR is basically analogous to Protobuf files. It has structs and tagged unions and so on.

Another technology from around the same time was DCE/RPC [2]. Microsoft adapted wholesale as MSRPC. Windows used it extensively around the time of NT 3.x for protocols like Exchange Server, and I believe it's still in wide use. DCE/RPC has its own IDL. You used the compiler to generate the stub implementations, just like Protobuf/gRPC.

Microsoft COM uses DCE/RPC under the hood, with lots of extensions [3]. CORBA emerged around the same time as DCE/RPC and COM and is roughly analogous in functionality.

COM and CORBA are explicitly object-oriented. While protocols like DCE/RPC and gRPC return values that are pure data, such as primitives and structs, COM and CORBA can return interfaces. An interface pretends to be a local in-memory instance, but its methods are "stubs" that invoke the underlying RPC call to execute them remotely. Methods can also return functions, which means you have whole trees of objects which are remote. Adding to that, both COM and CORBA use reference counting to hold onto objects, so if a client has received a remote object and reference counted it, the server needs to keep it around until the client either releases the refcount, or the client dies. COM and CORBA called this referential transparency, in that any object could be either local or remote, and a consumer of the interface didn't need to know about it. Of course, while nicely magical, this leads to a lot of complexity. I developed a rather complex distributed DCOM application in the late 1990s, and while it did, inexplicably, work quite well, it was also a nightmare to debug and keep stable.

While COM is alive and well in Windows these days (and interestingly enough, some APIs like DirectX use the COM pattern of defining interfaces via IUnknown etc., but are not actually true COM), DCOM turned out to be a mistake, and CORBA failed for some of the same reasons, although for many reasons unique to CORBA as well. CORBA made tons of design mistakes.

SOAP and WS-*, and of course XML-RPC and JSON-RPC, came later. The wheel has been reinvented many times.

[1] https://datatracker.ietf.org/wg/oncrpc/about/

[2] https://en.wikipedia.org/wiki/DCE/RPC

[3] https://learn.microsoft.com/en-us/openspecs/windows_protocol...

DCE RPC was the old Apollo Computer Inc RPC system probably with big fixes and support for 64-bit CPUs. I'm not sure what relationship it has to ONC RPC, peer, ancestor or descendant.

See: https://jim.rees.org/apollo-archive/papers/ncs.pdf