Both critical bugs are integer overflows. It's unclear to me why our languages still default to modulo arithmetic semantics. I feel Rust had a chance to fix this, but also dropped the ball.
> Rust had a chance to fix this, but also dropped the ball.
By default, a Rust project will panic on integer overflow in debug builds and will overflow on release builds. Two key points to note, however:
1. You can change the setting so that your project panics in release or overflows in debug mode.
2. We reserved the right to change the default at some point in the future. This will probably be widely communicated before it ever happens, and last I heard we are still waiting for the cost of performing those checks to be "reasonable" before thinking about making such a change.
Furthermore, because integer overflow is defined behavior, the integer overflow is never considered a root cause in Rust. In order for an integer overflow to express as UB in Rust, you'd have to use it in conjunction with an `unsafe` block that was failing to ensure its invariants, and that would be considered the root cause. If you're not using `unsafe`, then an integer overflow is at worst a logic bug.
A logic bug can be dangerous too though. E.g. Bumping a user ID, to get a "fresh" one or calculate port to open based on offset. When not bounded to a known range, this kind of logic can easily pose a serious security risk. Most of the time, it will probably just work, but under extreme conditions, it will fail. If your language at least catch the overflow and crash instead of wrapping around, you "only" have a denial of service.
Can imagine that implementing bounds checking can be costly, when done in software. Wonder if there are any hardware improvements that could reduce risk in this area.
Indeed, nobody ever said that logic bugs were good, but as a category of flaw it means that integer overflow in Rust isn't particularly interesting compared to all the other innumerable ways to introduce logic bugs. And I say that as someone who wouldn't really mind if the behavior was changed to panic-by-default in release mode.
If you identify an area as risky, it's trivial in Rust to do a checked_add or saturating_add. The challenge is obviously identifying this, but having easy library functions anectotally leads to people looking for it in code reviews.
A logic bug can be just as bad as any other kind of bug. Security bugs/memory corruption don't always deserve the extra special treatment they get, nor are they the only kind of remotely exploitable issue.
Just in case people don't know, you can get the same behavior with C or C++ by invoking GCC or Clang with -ftrapv. I don't know about Rust, but for C and C++ -ftrapv will only fault on signed integer overflow, as signed integer overflow is UB in C/C++ but unsigned integer overflow is well defined (it's guaranteed to wrap around to zero on all platforms). So even if you're trapping on integer overflow, there are still plenty of weird things that can happen if you unintentionally overflow an unsigned integer.
> I heard we are still waiting for the cost of performing those checks to be "reasonable" before thinking about making such a change.
What I don't understand is, checking for integer overflow is extremely cheap in hardware, so why is there any cost for performing those checks? What am I missing?
Part of it is that most CPU architectures don’t make overflow checking as cheap as it could be (there’s no option to trap on overflow, so you need a branch after every arithmetic operation, which has some cost). Another part is the compiler: a lot of compiler optimizations assume that arithmetic is a pure operation that can be added and removed and reordered as needed. So right now, adding overflow checks means opting out of a ton of optimizations. With care it may be possible to recover most of those optimizations, but it would require major improvements to LLVM.
This is a great point. But how about if the language would allow the programmer to specify what range of values is expected for function input/output?
Then a compiler could try to reason about the computation and decide that overflow does not happen if all values are within bounds, and just add checks at the function boundaries.
Integer arithmetic is a significant part of ~every program. A single branch that checks the overflow flag is not expensive. But branching on that flag every time you do integer math is death by a billion paper cuts.
Most architectures do not provide an interrupt that is generated by an integer overflow. Since this would be a significant architectural change in the hardware, it can't be simply added in.
Additionally, if you are running inside an operating system, handling an interrupt usually incurs a trip through the kernel, which would add extra overhead every time an overflow did happen. Since there's a lot of software which depends on integers overflowing, this overhead on each overflow could significantly impact legacy software.
Using a new instruction none of that would be an issue. And as someone pointed out, on x86 it wouldn't even be a new instruction, INTO already exists. But apparently it didn't make it in x64 because nobody used it :/
x86 at one time had a single-byte instruction that would trap if the overflow bit was set, INTO. It doesn't exist in 64-bit mode, I believe, and it was never widely used as far as I know. The performance implications of adding even a single additional instruction to every integer operation were probably still prohibitive? (And there's a history in x86 of specialized clever instructions and mechanisms going unused, due to being slower than doing the same thing some other way.)
Presumably the program has to check bit in a status register or something like that to tell if the previous instruction caused overflow, no? That means an extra branch after each arithmetic instruction. I imagine that's not cheap?
Fascinating. Haven't had a chance to do Rust yet, but I think I would change this so that they were consistent. I do embedded, and that kind of "behaves differently in different places" is the worst kind of bug to figure out.
It doesn't behave differently in different places, it differs based on build profile. You test in debug mode, which is where overflows will panic, which makes them quite obvious. If you want to pay the price of overflow checks everywhere in release mode, then you can turn them on there as well (it's not that much of a performance penalty, but that might not be true on embedded...). It's effectively just a compiler flag.
Our processors also require manually manipulating registers.
The whole point of higher level programming languages is to abstract away the fiddly bits of dealing with processors that we don't want to have to deal with.
In this case it's really that the cost of determining if an overflow did occur or will occur on modern architectures is too high and the likelihood too low for it to be reasonable to perform the checks in most cases in native code. Might be different for interpreted languages depending on a lot of things (whether or not they even use integer arithmetic, whether or not they default to some arbitrary precision integers by default, etc.). If common architectures automatically interrupted on overflow rather than setting a flag at no additional cost, I'd think you'd see safety guarantees instantly.
Unsigned integers in C are defined to wrap on overflow, so they shouldn't be affected by what the underlying processor wants to do. And just because signed overflows are undefined doesn't mean you get what the processor provides either.
So basically, don't forget to test with UBSan.
(I don't like -fno-strict-overflow as a solution, because defining incorrect behavior isn't much better than undefined behavior.)
TIL modulo wrapping for unsigned ints is actually in the standard, I could have sworn it was implementation defined (as opposed to signed wrapping, which is undefined).
error: this arithmetic operation will overflow
--> src/main.rs:2:18
|
2 | let a: u64 = u64::MAX + 1;
| ^^^^^^^^^^^^ attempt to compute `u64::MAX + 1_u64`, which would overflow
|
= note: `#[deny(arithmetic_overflow)]` on by default
Rust also allows for overflowing arithmetic (preserving the default to fail):
There are times when you want to know how much overflow occurred -- think of the way you learn to do multi-digit addition. There is a checked_add that returns an Option<T> if you only care about success/failure.
> If A happened to be larger than C, and therefore the result of the subtraction underflowed, then we can correct for this by adding p to the result. [...] we can multiply B by 2^32 using a binary shift and a subtraction, and then add it to the result. We might encounter an overflow, but we can correct for that by subtracting p.
(Highlighting the parts that relate to reacting to overflow.)
The `(T, bool)` that gets returned is friendly towards optimization.
If you don't use the bool, the overflowing arithmetic is reduced to efficient arithmetic which is overflowing by default. If you do use the bool, the generated code contains one extra instruction.
As a sibling notes, currently, this is for debug builds. So, if you change that playground to "Release", you'll see it wrap.
(I love this feature, and I wish they had done it in release mode too. The sibling comment has some notes on that, too.)
(But, e.g., were `git` written in Rust, presumably the end product would be a release build. Now, you can enable the check there, but that is something you have to do, today.)
(But also note, that, in all cases, it's well-defined. Vs. C, where some overflows are UB.)
Rust makes integer overflow panic in debug builds, so Rust code is effectively required to opt into overflowing operations for correctness reasons. It disables those checks on release builds for performance reasons, but as sibling comments point out, it reserves the right to change that behavior.
Unfortunately, there is a circular dependency here. Languages are reluctant to make integer overflows error conditions because there is a moderately high overhead to checking overflow conditions constantly, and processors (and compilers) are unwilling to make overflow checks cheaper because they benchmarks they care about don't do such checks.
That sounds like the similar, but opposite case of tail recursion optimization. Some languages/compilers don't do it because devs want stack traces. But allow TCO in and now the code that gets written is quite different than the code that would not do tail calls because TCO doesn't exist.
Also a surprising amount of undefined behavior gets relied on in code. I don't use Rust, but the idea that they could potentially change the future behavior on overflow seems... risky?
I'm wondering if what you are asking for is what Swift does - overflow kills your program, but you can opt into allowing it by using "Overflow Operators" (&+, &- and &*).
This crashes in Swift
var potentialOverflow = Int16.max
potentialOverflow += 1
This does not crash
var potentialOverflow = Int16.max
potentialOverflow &+= 1
As others have mentioned, the default overflow behavior in Rust can be configured to panic. To explicitly wrap around on overflow in every configuration, you can use the newtype wrapper Wrapping, or use the wrapping_add(), wrapping_mul(), etc. methods on the basic integer types. There are also variations such as saturating_op(), checked_op(), and overflowing_op() to detect the overflow and handle it appropriately.
Sure, I was suggesting that what the OP is asking for is not what Rust does, but what Swift does. This is not a configuration thing, if you don't want a runtime exception on overflow, you must use a different arithmetic operator.
Swift's behavior comes at a cost - it is not exactly the fastest language out there ;) Another no-overflow oddity is that Swift doesn't have a rand() equivalent. You can't get fast psuedorandom numbers in Swift unless you are on the Mac, in which case you can import GameplayKit and get gaming-appropriate pseudorandom numbers.
EDIT - to be clear, I am not suggesting that anyone change their own chosen programming language. But if you'd like, install Swift on your dev machine and make a Swift implementation of the critical section of your Rust code. Debug, optimize, tweak, etc. And you'll get a pretty good idea of what kind of performance you have to give up to do what many people are asking :)
I'm quite paranoid about integer overflows, so in my hobby projects I now have a habit of always using helper functions (which generate an error on overflow) instead of "bare" math operators, and whenever I see a bare math operator without any checks in an open source project (and from what I've seen almost no one checks for overflows) I wonder whether they thought about potential consequences or I'm being too paranoid
There are different ways to design it, but a simple version could be a function that takes 2 operands and 1 result pointer as inputs, and returns boolean (true if success, false if it overflowed):
bool SafeAddIntInt(int32_t x, int32_t y, int32_t *r);
so the caller could say
int32_t result;
if (SafeAddIntInt(x, y, &result)) {
// do something with result
} else {
// handle overflow
}
An even simpler version could just abort on over/underflow.
Note that there is no such thing as integer underflow. INT_MIN-1 is an overflow just as much as INT_MAX+1.
Only floats can suffer from underflow, which happens when you want to represent a number whose absolute value is smaller than the floating point precision can allow (e.g. trying to represent 1/2^32 in a 32-bit float).
Because saturating math is not "more right", just "different wrong". The "right" way of checking an error condition after every integer operation is prohibitively expensive.
From the language side, what I wish for is a sort of NaN for integer operations. I would not want to check for overflow on every operation, but I would want to know after a couple of them if somewhere an overflow had occurred. On the hardware side this could be done with a sticky overflow bit, which some architectures already support.
I think the ball is on the hardware side and in my opinion Rust did the most sensible thing possible with contemporary hardware.
Integer overflow isn't a security issue unless your program's memory safety depends on the correctness of the integer operation. Safe rust doesn't (in any build mode), but C/C++ does.
> Integer overflow isn't a security issue unless your program's memory safety depends on the correctness of the integer operation.
That's simply not true and has wide-reaching horrible effects that can occur. The wrong number of tickets can be purchased from a website, charging for less than were purchased. The DNR order can be put in place instead of SAVE LIFE. There are countless security issues that can occur.
Saying that integer overflow is only an issue for memory safety is really bad and incorrect advice.
That's most logic issues though, is it not? I agree with you though, i wish Rust more commonly pushed "safer" (not in the UB way) code, like `Vec::get` and `u32::overflow_add` and etc.
Luckily lints help to easily ban the arithmetic/etc ops from projects. Nevertheless i feel it should be a bit closer to Rust's home.
Not via a compiler flag, no. The way to "opt out" of bounds checks is to replace `foo[bar]` with `unsafe { foo.get_unchecked(bar) }` at a given callsite. And the use of `unsafe` is going to immediately raise the eyebrow of any code reviewer or auditor.
Sure, but a logic error is a fundamentally different class of error compared to a memory error. The potential harm of a logic error is limited in scope to what the program was written to be able to do. A memory error can lead to arbitrary code execution.
Absolutely, and I didn't suggest otherwise. But a logic error generally won't lead to your entire system getting pwned unless the program that has the logic error is one for something like user administration or managing a database etc.
Don't see how. Given the hardware Rust is designed to program you have to compromise some or all of efficiency, memory usage and complexity to solve overflow.
Rust can guarantee some things about collections which are not possible in C, so a lot more range checks and overflow checks could be omitted. Together with actually having the saturating/overflowing/checked adds, this makes the whole thing a lot safer and easier to deal with where you need to.
Great. Tell bbojan how Rust didn't actually "drop the ball" then. When you do you'll be told about all the different ways Rust doesn't actually solve overflow. And those claims will likely be correct because they're self evident.
My point -- my sole point -- is that Rust is like it is because none of the alternatives are viable for Rust; Rust must run efficiently (as a "systems" level language use defines "efficient") on hardware that silently wraps words. Rust can't fix that and still be Rust. There is no ball to drop.
I wonder if using 64-bit integers all over the place would alleviate this a bit. If your integers represent some real quantities (sizes of objects, etc), the sizes have to be unrealistically huge to trigger an overflow. If your integer is a counter, it would take years to increment it in a tight loop to achieve an overflow.
The cost of operating on 64-bit integers is about the same as operating on 32-bit integers on most modern CPUs (except maybe 32-bit cores in MCUs).
By default, a Rust project will panic on integer overflow in debug builds and will overflow on release builds. Two key points to note, however:
1. You can change the setting so that your project panics in release or overflows in debug mode.
2. We reserved the right to change the default at some point in the future. This will probably be widely communicated before it ever happens, and last I heard we are still waiting for the cost of performing those checks to be "reasonable" before thinking about making such a change.