[holy_city]

This is similar to the work done in winapi [1], com-imp[2] and my own tangential work on porting VST3 [3] to Rust [4]

I'm really glad MS is doing this. What needs to be a bit clearer to me is how they maintain ABI compatibility under the hood of MSVC for COM interfaces (which uses the vtable layout of an inherited class) and how that's compatible with MinGW/GCC stacks on Windows, mostly what can break it. I got stuck porting VST3 with multiple inheritance, and it was a headache trying to reverse engineer the appropriate struct layouts for COM implementations.

[1] https://github.com/retep998/winapi-rs/blob/0.3/src/macros.rs

[2] https://github.com/Connicpu/com-impl

[3] https://github.com/steinbergmedia/vst3sdk

[4] https://github.com/m-hilgendorf/cli-host (sorry for the messy code, it was a weekend of hacking away trying to host a VST3 in pure Rust)

[barrkel]

COM is an ABI standard. The structs are defined in terms of C with Winapi (stdcall) calling convention. Very little needs reverse engineering - it's all pIntf->vTable->func(pIntf, ...). You can explain it on a whiteboard in a couple of minutes.

The way MSVC does it may need reverse engineering (it may be patented btw). I could explain how Delphi implements COM interfaces, but any specific implementation is actually more complicated than the ABI, because they're trying to add implementation ergonomics on top of the basic calling convention.

[holy_city]

Is the ABI documented anywhere? Every time I google around for it, I just get information like "COM is ABI stable and language agnostic" but not what the ABI is. I've successfully implemented implementations of single COM interfaces and get the basics, my trouble was in implementing many interfaces for the same implementation and running into copious segfaults when testing the Rust implementation through a reference app written in C++.

[pjmlp]

Here are some good COM books.

Essential COM by Don Box

https://www.amazon.com/Essential-COM-Don-Box/dp/0201634465

Inside COM by Dale Rogerson

https://www.amazon.com/Inside-Microsoft-Programming-Dale-Rog...

Then if you have access to public libraries, maybe one of them has one of the several Microsoft Systems Journals issues, later MSDN Magazine, with plenty of low level COM articles.

COM is from the days where good documentation was to be found in books, not on the Interwebs.

[teh_klev]

+1 for Essential COM by Don Box, it still survives my bookshelf purges..."just-in-case". IUnknown and IDispatch are burned permanently in my memory from a period of my life building a COM/CORBA bridge.

[alxlaz]

I'm gonna piggyback on your comment to give another shout out to Essential COM. I haven't touched COM in ages but that book is so good I still pick it up every once in a while -- and it was a lifesaver back when I did touch COM on a daily basis.

[taude]

Wow, that's a throw back to the late 90s, early 2000s. I'm still slightly scarred from working with ATL and COM. A lot of people around here would probably be surprised to hear that there were even Python -> COM bindings back in the day that were even used to ship server software once upon a time. Anyway, I remember that Don Box book well.

[ptx]

Not only "back in the day" – those COM bindings for Python were most recently updated last month: https://pypi.org/project/pywin32/#history

I highly recommend using them if you're ever forced to work with VBA and need to implement some non-trivial functionality.

[contextfree]

There are also new Python bindings for WinRT (which is based on COM): https://github.com/Microsoft/xlang/tree/master/src/package/p...

[7thaccount]

Yea...Python and Powershell are the only languages that I know of outside C#/VBA/VB.NET with decent COM support. I hate COM, but it works.

[crtlaltdel]

oh how i wish i knew this back when...or just looked harder i guess ;)

[holy_city]

thank you!

[vic20forever]

archive.org has the following pre-dotnet edition of MSDN Library - you'll find a lot of COM info in it:

https://archive.org/details/MSDN_Library_October_2001_Disc_1

https://archive.org/details/MSDN_Library_October_2001_Disc_2

https://archive.org/details/MSDN_Library_October_2001_Disc_3

[barrkel]

What's to document? The stdcall / WINAPI calling convention, which is of course OS and architecture dependent, but can be summarized as on the stack, right-to-left, callee pops.

The rest is just that interfaces are doubly indirected to get to a vtable (an array or struct of function pointers), the method is chosen by ordinal in the vtable (first three are always the IUnknown methods), and the interface is passed as the first argument.

How you construct those vtables and how you select one to return in QueryInterface, and how you implement interfaces (i.e. traits) so you can convert them into a vtable is where all the work is. You can do anything you like that works as long as it's called according to the COM conventions.

Delphi works by implementing each method in the vtable with a stub which subtracts the offset of the vtable pointer in the instance data from the passed in interface, and then jumps to the implementation method on the instance after the instance has now been adjusted. Instances look like this:

    [class pointer, the native vtable] <- normal instance pointers
    [vtable interface 1] <- COM interface pointers
    ...
    [vtable interface n] <- COM interface pointers
    instance field 1
    ...
    instance field n

So you can see that in order to convert a COM interface pointer into an instance pointer that the methods expect, the COM interface pointer needs to be adjusted depending on the offset of the vtable in the instance.

In a language with multiple inheritance like C++, the compiler vendor targeting Windows may choose a layout which is suitable for COM's calling convention (there's more than one way to do MI, e.g. fat pointers is another way to go that wouldn't be COM-compatible). If the vendor does that, then they make implementation of COM interfaces much easier. And if they don't, well, life isn't going to be easy. Technically you could do a bunch of stuff with reflection and code generation, but that's harder and harder these days with security restrictions around code writing code. You could write macros which create statically initialized structures of the right shape, and fields of the right type, to emulate the same effect as Delphi's scheme as I sketched above, or some other method which would work with COM's calling convention.

[_fbpt]

Possibly relevant: Wine (Win32 implementation on Linux) has an IDL compiler, which compiles IDL (language for defining COM objects) into C: https://www.systutorials.com/docs/linux/man/1-widl/

[snagglegaggle]

Realize that COM is supposed to be used with code generators. You can write it in raw C even, but it quickly becomes intractable. Perhaps I did not read completely but this is something I see missing from the post -- the annotations fit what the pre-pass C++ code looks like, but don't mention that layer?

[sayrer]

You may want to look at XPCOM. At one point it interoperated with MS COM, and it might still.

[ChrisSD]

> The structs are defined in terms of C with Winapi (stdcall) calling convention.

This isn't quite right. The calling convention is a Windows C++ variant of stdcall. See this issue: https://github.com/rust-lang/rfcs/issues/1342

However, 99% of the time the difference won't be an issue.

[pjmlp]

COM layout is followed upon most mainstream Windows compiled languages, namely major C++ compilers, .NET, Delphi, Eiffel, Ada, so it is not MSVC++ keeping ABI compatibility under the hood on their own.

[holy_city]

My issue isn't ABI stability but the ABI itself w.r.t vtable layout. Best I can tell it should be similar to Itanium's spec? [1]. It's been months since I did this, but iirc my problems stemmed from having multiple interfaces on top of the same implementation and the ordering/layout of those interfaces, though the IUnknown interface which is supposed to handle that.

[1] https://itanium-cxx-abi.github.io/cxx-abi/abi.html#vtable

[barrkel]

COM is agnostic as to how you do multiple inheritance - it doesn't have the concept. It specifies the QueryInterface protocol, but you don't need to return the same instance for the result of the QI call, just one that uses the same lifetime refcount.

Tear-off interfaces and delegated implementations are things in this world.

[ptx]

Meaning that the AddRef/Release counter has to be shared between that group of objects?

[pjc50]

Same object, different interfaces.

[magicalhippo]

You can implement the other interfaces using other objects, but they should delegate the ref counting to the root/main object.

[jcranmer]

MSVC uses a completely different ABI from Itanium, and you shouldn't rely on the Itanium ABI to inform you what it might look like.

vtable layout in the most basic situations is going to be accidentally portable because those situations boil down to "it's a struct of function pointers," and there's only so many ways you can order the fields of a struct. But even here, MSVC uses a quite different ABI: the order of the vtable entries can change if you overload a virtual method with a non-virtual method.

[spacechild1]

AFAIK, non-virtual methods never affect the vtable layout. But when you overload a virtual method with another virtual method, the ordering in the vtable is unspecified!

Also, a public COM interface mustn't have a virtual destructor, because some compilers (e.g. recent GCC) put more than one method in the vtable. Implementation classes might define a virtual destructor, though.

[Const-me]

If you can read C#, maybe this will help: https://github.com/Const-me/ComLightInterop

Specifically, this class implementing vtable callable from C++: https://github.com/Const-me/ComLightInterop/blob/master/ComL...

About multiple interfaces, all of them need 3 first vtable entries pointing to the 3 IUnknown methods. Also, don't forget that when client calls QueryInterface on any interface of the same object with IID_IUnknown argument, you must return same IUnknown pointer. Some parts of COM use that pointer as object's identity.