With C++, if you have char*'s (because you don't need to own the memory) and you pass it to a function that takes a const std::string& (because it also doesn't want to own the memory), then there will still be an implicit conversion to a temporary std::string (involving an allocation) despite neither the caller or the callee needing to own any memory.
With Rust, if you have a &str (because you don't need to own the memory) and you pass it to any function that takes a String (or even the unidiomatic &String), then you will get a compile error. There won't be any implicit conversion of types and therefore no implicit allocation. If you really want to pass it, you need to explicitly convert it, making the cost of the allocation explicit.
Rust's "too many strings" model says "there are many different ways in which you can use string-like objects, each with their own performance tradeoffs. Know which one you want to use in your code or I won't compile".
This discussion is making me wonder if windows-rs [1], the crate with official Rust bindings for all Windows APIs, is doing something that's not idiomatic Rust. Specifically, for any Windows API function that takes a UTF-16 string as a parameter, the signature for that parameter is something like "impl IntoParam<PCWSTR>". The crate then implements that trait for String and &str, so you can pass a normal Rust UTF-8 string (even a string literal), and it'll be automatically converted to a freshly-allocated, null-terminated UTF-16 string (which gets freed after the function call). That seems like it could lead to the same thoughtless inefficiency as in the story about the Chrome omnibox.
Well that will be necessary until windows gets UTF-8 APIs. Probably not soon. Until then there are various optimizations you can do, like caching the UTF-16 conversion alongside the UTF-8 string (good for calling OS APIs frequently with with long-lived strings), allocating temporary UTF-16 conversions on the stack (good for infrequent calls with strings up to a certain size), or storing raw UTF-16 strings as opaque bytes in Rust memory (good for providing strings back to the OS that you got from the OS).
You should try to avoid calling OS APIs in general and cache the results as much as possible. Who knows what the performance characteristics are of an API that has to serve 7 layers of historical OSes simultaneously. Unless you're directly interfacing with the kernel you shouldn't expect much. Omnibar-like layered calls between your app and the OS are a worst-case scenario regardless of conversions.
Very interesting I wasn't aware. After glancing over that doc, it looks like they smuggle UTF-8 in through the -A variant windows APIs [1] by explicitly setting the CP_UTF8 codepage in an application manifest. I wonder if this actually uses UTF-8 internally to service the API call or if it just manually converts strings to wide form and calls the -W variant on the windows side instead of making you do it on the app side. If the latter it may be better to avoid this feature so you don't close the door on potential optimizations like I mentioned above.
[1]: Windows has two variants of many API calls with either -A or -W suffix, where the -A suffix is for strings formatted as 1-byte ASCII (or a specified codepage) and the -W suffix is for strings formatted as 2-byte UTF-16 (kinda). Example: DlgDirListA / DlgDirListW, https://docs.microsoft.com/en-us/windows/win32/api/winuser/n...
It would most likely suffer from similar problems when interacting with the C and C++ APIs in the rest of Chrome though (e.g. what to do if you have a Rust String, but the other side wants a const ref to a C++ std::string).
My minor, unpolished grievance with Rust's approach is that you have to do this for all kinds of types (e.g., Path vs PathBuf). It's tedious to have to write these pairs all the time, along with all of the trait implementations and so on. It almost feels like it would be nice if the type system could allow us to write `String` or `PathBuf` and automatically generate the corresponding `str` or `Path` types.
> With C++, if you have char*'s (because you don't need to own the memory)
If you are using C strings in C++ you are either doing something incredibly low level or don't care about performance at all. C strings require strlen calls or something equivalent for basic operations and you can easily run into code with exploding runtime if you aren't extremely careful.
A variant of what is becoming string_view in the standard has existed within Google's codebase(s) for years. I don't recall using it much in Chromium when I worked in there, but it's all over Google3 and is now in absl (Google's open sourcing of some of its base c++ components).
The point is not that c++ can't do this (I also have code that does this dating back over 10 years), it's that despite having code to do string_view/string_piece, Chromium was still performing 25,000 allocations per keystroke in its Omnibox because c++ has other common ways to represent "constant string owned by someone else", and there are hidden performance issues that will trip up even experienced programmers when mixing these ways incorrectly.
Despite having better options available (either in the standard library or custom code), the less optimal ways still get used.
Rust had the benefit of learning from c++'s mistakes and separated the concepts of owned vs unowned strings in to separate types with explicit conversions required whenever an allocation would occur. This was baked in to the language from the beginning and so you don't get a mix of different types in signatures to convey the concept of pointing to a slice of a string owned by someone else, you just have &str.
Even if you get fancy with your interface and do things like AsRef<str>, there's still no concern about implicit or hidden allocations. Any time you need to own the memory (either for yourself or to pass in to another function) you need to do so explicitly and you end up with a different type (much to the chagrin and confusion of newcomers to the language).
C++ is trying to correct its mistakes also, but not everyone can use those latest features and even if they can, the mistakes still have to be left in for compatibility reasons.
Technologically, Rust's only built-in string type, &str, is a reference to a string slice - that is, you can't change it (the reference isn't mutable) and it is both a pointer to the start of some UTF-8 and the length of the UTF-8.
What encoding? Always UTF-8. Only UTF-8. Not "Well, it's kinda UTF-8 but..." it's just always UTF-8. This moves the burden to a single place, your text decoding code, to do things correctly, and great news - the entire world is moving to UTF-8, so you're on a downhill gradient where every week this works better without you lifting a finger.
That reference knowing the length is brilliant. Trimming whitespace off a string? You can just make another immutable reference to the smaller trimmed string. Zero copies. Slicing a URL up into components? You can do that too, zero copies. And yet it's all memory safe.
Now, Chromium is not some raw firmware for a $1 micro-controller, so it has library types like Rust's alloc::string::String (you can just name it "String" in normal Rust code but that is its full name) which, as its presence in alloc suggests, is an allocating String type, you can concatenate them, you can make them by formatting a bunch of other variables, the default ones are empty, the data goes on your heap and so on. But, String is AsRef<str> which means if what you've got is a String, and what you're doing is calling a function that wants &str Rust is OK with that and it costs nothing at runtime. Why? Because that &str is just two of the elements of the String type you had, the pointer into the heap and the length, it's easy.
Rust has lots of other types for stuff like Foreign Interfaces, like CStr and CString (for the C-style NUL-terminated array of bytes which might be text) but your pure Rust code shouldn't care about those, often it can say (unsafely) "Look, the C++ promises this is UTF-8, we'll take their word for it" or "I only need it to have bytes in it, let's make [u8] and we're done".
Culturally, Rust programmers write &str when that would do. There's a strong cultural pressure not to write String when you really mean &str, and the compiler won't let you write &str if you needed String. So this results in less thunking of the sort complained about in C++
In C++ when I see `DoX(y)` I have to worry every time about temporary lifetimes, copy vs move operator, and a bunch of other things that are easy to miss during code review. It is so easy to accidentally copy large strings around many times in a performance critical loop.
Rust makes all of that easier to see during code review. It is very explicit about these things.
I'm a Google employee working on chromium and chromeOS and have been asking internally about rust support for over a year now, so it's exciting that it's making progress.
It's a complicated but well-thought out system which tends to avoid copies by making them explicit in the source code and preferring taking references or slices which are cheap operations.
The string slice for example is an Unicode-capable view into bytes of the string (immutably pre-compiled static bytes in the binary, bytes of fixed length on the stack or heap-allocated). The aliasing rules are enforced by the compiler, so it is safe to throw around pointers and sizes and not to worry about buffer overflows, as long as it compiles.
Everyone’s trying to justify the response when the honest truth is that no, Rust doesn’t solve the problem of abstraction layer impedance mismatches causing ownership to be dropped only to be reacquired at the next level. On a sufficiently large/complicated code base, the problem will arise.
As others have mentioned, various kinds of string types are baked into the language which makes it ergonomic to do “the right thing” from the get go, but hard to say. I would be skeptical of claims that it would make a difference, especially in the interim where you now have an added impedance mismatch with C++, Rust, C.
> Rust doesn’t solve the problem of abstraction layer impedance mismatches causing ownership to be dropped only to be reacquired at the next level
The expression of what C++ does here in Rust is awkward and, I think, nobody has proposed it because you'd never write that. Basically C++ char* is a raw pointer. Rust does have those, but you'd never use them in this context.
What you would use is either the borrowed slice reference &str or the owning String type, but in both cases we have an owned object and there's our crucial difference. If you've got the owned String, and I needed an owned String, I should ask for your owned String, and we're done.
In C++ "dropping" ownership as you describe is no big deal, the C++ design doesn't care, but in Rust if you actually drop(foo) it's gone. The references to it can't out-live that, if it's gone then they're gone. If you write code that gives away references and then tries to drop the thing they're references to, Rust will object that this is nonsense, because it is nonsense, you need to ensure those references are gone before dropping the thing they refer to.
As a result I feel you're greatly under-estimating the ergonomic difference.
> In C++ "dropping" ownership as you describe is no big deal, the C++ design doesn't care, but in Rust if you actually drop(foo) it's gone
I think you’ve built a straw man of my argument and then argued with that.
Clearly I meant that it seems possible that a sufficient complicated call stack could still be set up to jump between needing the owned String type and the borrowed &str type. That’s what I meant by dropping ownership as that’s what’s happening in the c++ code when you go between char*/string (the API is dropping its need for ownership). The argument of “ If you've got the owned String, and I needed an owned String, I should ask for your owned String, and we're done” is weak because that same argument would apply to C++ code and yet the code still ended up that way when you pasted together components in a very large code base. Now maybe it’s a bit simpler because you have string, string&, const string&, and const char* and doing that antipattern that happened in C++ just wouldn’t be ergonomic in Rust. Maybe. But that feels like a very thin argument and not “this is impossible in Rust”.
I am definitely not arguing that it's impossible but my experiences with Rust lead me to think you've significantly underestimated how important those ergonomics are.
The "I should ask for your owned String" argument does not apply equally well in C++ because of a crucial design infelicity in C++. Your caller may well not have an owned std::string.
In C++ raw char pointers are totally a thing. Because std::string is a late addition (if you learned C++ in the early 1990s a "string" class was maybe an interesting exercise, not a library type) the string literals aren't a built-in string type, and much of the API isn't shaped for such a type either.
Now, the effect is those are (sometimes) owning pointers, it is possible I own some C++ string in this sense, and all I have is a pointer into it. If I give you that pointer, it's not because I didn't give you the owned string, that pointer is my owned string. You want a std::string and there's no reason I would have one at all.
You can mutate these strings, but of course you can't extend them because you've got no way to know how to communicate with the allocator, maybe they live on the stack, or in a private heap. At the time this seemed like a good idea, today we don't think so.
How is this solved in Rust though? The only thing Rust solves is that you don't accidentally hang onto a bad reference, but I'm failing to see how &str/String is meaningfully different from char/string since the same issue applies (char = borrowed, string = owned). My Rust is rusty so apologies for the pun & any syntax errors. Let's say you have the following:
// some process managed by team a
fn caller1(...) {
String s;
level1Callee(&s);
}
// some process managed by team b
fn caller2(s: &str) {
level1Callee(s);
}
// library 1 by team c
fn level1Callee(iDontNeedOwnershipOrDoI: &str) {
level2Callee(iDontNeedOwnershipOrDoI.to_owned())
}
// library 1 by team d
fn level2Callee(iNeedStrongOwnershipBecauseIMutateTheStringAnyway: String) {
level3Callee(&iNeedStrongOwnershipBecauseIMutateTheStringAnyway)
}
This is roughly what happened in Chrome as I understand it (except multiple times because of independent libraries that didn't notice that they probably should have just made a copy to begin with). Let's pretend the codebase had been written originally in Rust. How does Rust avoid this problem from coming up? This didn't happen in Chrome because of ownership. It came up organically because of years of refactoring obfuscated things. For example, level2Callee started out not needing strong ownership but then started calling a library that did (refactoring a complex codebase is very hard & time consuming). Rinse & repeat after many years. Now maybe Rust tooling is better able to point out the unnecessary acquiring/dropping of the strings but that seems unlikely - the problem is statically very difficult to lint around.
Chromium had a few problems, they're public so we can go read the changes made and look at the context.
In a lot of places in Chromium the impedance choices are arbitrary. You have a raw pointer but need a string or vice versa. Ownership isn't the problem, so in Rust you literally just always choose &str for these APIs and pay nothing. A team who design their API taking &String in this situation get the same treatment as a team who name all their types Data1, Data2, Data3 and so on. Somebody senior fetches the water spray, "No. Bad programmer".
You might be outraged, surely C++ programmers also never get this wrong. But nope, happens all the time as Chromium illustrates.
Added: One cause that shows up in my review is this:
C++ strings know how long they are. The raw pointer does not. As a result if we have lots of people asking if their thing is "some text" it's tempting to demand they give us a string, since if the string's length isn't 9 we don't need to look at the text itself. It's an optimisation! Rust's &str knows how long it is.
With C++, if you have char*'s (because you don't need to own the memory) and you pass it to a function that takes a const std::string& (because it also doesn't want to own the memory), then there will still be an implicit conversion to a temporary std::string (involving an allocation) despite neither the caller or the callee needing to own any memory.
With Rust, if you have a &str (because you don't need to own the memory) and you pass it to any function that takes a String (or even the unidiomatic &String), then you will get a compile error. There won't be any implicit conversion of types and therefore no implicit allocation. If you really want to pass it, you need to explicitly convert it, making the cost of the allocation explicit.
Rust's "too many strings" model says "there are many different ways in which you can use string-like objects, each with their own performance tradeoffs. Know which one you want to use in your code or I won't compile".