[foxhill]

apologies, perhaps i’m missing something here, having not used cap’n proto in any context at all before.

is it not possible to delete the rvalue reference overload of ‘getList’?

as far as i can tell, the error producing code wouldn’t have produced a diagnostic, but failed to build in the first instance, like the rust case?

[kentonv]

That would catch some legitimate use cases, where you get the list and immediately use it on the same line. Admittedly this is not so common for lists, but very common for struct readers, e.g.:

    int i = call.send().getSomeStruct().getValue();

Here, even though `send()` returns a response that is not saved anywhere, and a struct reader is constructed from it, the struct reader is used immediately in the same line, so there's no use-after-free.

Someone else mentioned using lifetimebound annotations. This will probably work a lot better, avoiding the false positives. It just hadn't been done because the annotations didn't exist at the time that most of Cap'n Proto was originally written.

[kentonv]

Oh actually there's a much more obvious case where prohibiting getters on rvalues would be a problem. It would prevent you from doing this in general:

    myReader.getFoo().getBar()

Here, `myReader` is already a view type; ownership of the backing buffer lives elsewhere. `getFoo()` returns a reader for some sub-struct, and `getBar()` returns a member of that struct. If we say getters are not permitted to be called on rvalues, this expression is illegal, but there's no actual problem with it and in practice we write code like this all the time.

[foxhill]

i could be wrong, but i’m reasonably confident that this is UB for even trivial types? someone more knowledgeable with the language lawyering would need to opine one way or the other.

regardless of that outcome, i think i’d prefer to require a value preserving the lifetime of the reader/view. in the cases that it may not be necessary, i'd prefer to lean on the optimiser to take care of it..!

[kentonv]

What's UB about it? Any temporary objects constructed during the evaluation of a statement live until the end of the statement. The standard is clear on that.

> i think i’d prefer to require a value preserving the lifetime of the reader/view. in the cases that it may not be necessary, i'd prefer to lean on the optimiser to take care of it..!

We'd all prefer APIs that cannot be used unsafely but realistically there's no magic the optimizer can do to make the problems with refcounting go away. You need to use a language like Rust to solve this.

[foxhill]

ah, sorry, i didn’t read that correctly.

perhaps for values like this you’re fine. i think my point still stands about the reader of a built-in list/sequence type, surely?

and, not to sound facetious, that’s exactly what optimisers do :)

the c++ type system is more than capable about reasoning about lifetimes, the issue is that, with c++, it’s an optional part of the language. also, the lack of non-destructive moves. but to require both of those things in the language would require, essentially, the borrow checker in rust.

[kentonv]

Unfortunately the C++ compiler cannot reason about much of anything as soon as you make a virtual function call, or even a call into a separate translation unit (unless maybe you are using LTO but that has its own issues).

E.g. if you do:

    {
      auto foo = std::make_shared<Foo>();
      bar->baz(foo);
    }

The compiler has to know what `baz()` does in order to know whether it can elide heap allocation and refcounting of `foo`. `baz()` could, after all, add a refcount on `foo` and keep it somewhere.

If `baz()` is virtual, or just implemented in a source file that the compiler cannot see at the time of compiling the calling code, then there's no ability to optimize at all. Even if the compiler does know the full implementation of `baz()`, eliding the heap allocation is not going to be easy. Maybe if `baz()` is very simple, it can do it? I actually don't know if the compiler is even capable of this when using shared_ptr.

Of course you can always say "well a sufficiently smart compiler could reason about your whole codebase including every implementation of a virtual call" but we program to the compiler we have, not the one we want. And frankly, if you had a compiler that smart it would be able to detect your use-after-free bugs and warn about them, so you wouldn't need to use shared_ptr everywehre.

[foxhill]

i am quite familiar with compiler internals :)

of course, across the TU boundary things get difficult, but i don’t think it’s fair to dismiss LTO entirely (although.. i agree with the thesis that it’s not particularly.. good)

similarly, de-virtualisation is an optimisation technique compilers will aggressively use to improve performance, although you’re right that it can’t look through another source file, so it is not without limitations here.

but we’re not being general, we’re being specific; the safety issues that are being discussed are well within the remit of the c++ type system here, and i don’t think we’re doing any favours to anyone by letting this rvalue be accessed in this way. it is certainly not idiomatic to provide library code that can so violently implode with seemingly regular use. i find it difficult to believe that lifetime issues like this are undiagnosable.