[rsp1984]

Fusing video with IMU for navigation is called VIN (visual-inertial navigation) or VIO (visual-inertial odometry) and the field has made enormous progress over the last 10-15 years. It's the same technology that the iPhone uses for all its AR features.

Dropped frames are one of the easiest things to handle. Yes, the visual feature tracking depends on frames of video but even cheap phone IMUs these days are good enough to dead-reckon for a second or two, especially when embedded into a sensor fusion framework, so the prediction errors resulting from a single lost frame should be very minimal and not enough to throw off the tracking.

That's why I find it hard to believe that the VIN in use by the Mars Helicopter (part of a multi billion dollar program) wouldn't be able to deal with a dropped frame. It just doesn't add up. I suspect that the situation is much more complex than what the article suggests and that more things went wrong than just a dropped camera frame.

[app4soft]

> It's the same technology that the iPhone uses for all its AR features

iPhone was not a first.

"Agent V" (2006) — a mobile AR game preinstalled on Nokia 3230 already has such technology.[0]

[0] https://news.ycombinator.com/item?id=20957157

[techrat]

1) He didn't say iPhone was first.

2) Was this something you even really needed to "Well, ackshully..." anyway?

[app4soft]

> 1) He didn't say iPhone was first.

I'm not argue. I'm just saying.

[ncmncm]

It means that the people who know most about closed-loop vehicle control were not involved in the design of the copter. Such fragility is a really elementary design mistake any experienced engineer would not make. Certainly the vehicle that delivered the lander would not suffer from the same mistake.

My interpretion is that the copter, as an inessential system component, was seen as an opportunity for junior people to get some end-to-end experience. I hope they are learning the right things.

[ehsankia]

The article makes it sound like after the missed frame, every subsequent frame had the wrong timestamp and was processed wrong, so it sounds like more of a bug. If the subsequent timestamps had the right timestamps, and the calculation was based on delta timestamps, then it would basically basically interpolate over the missing frame and then recover. The fact that every subsequent frame was assumed to be 33ms in the past was the issue here.

[mikepurvis]

And that being a system boundary thing, it’s easy to imagine it as an integration issue, where the controls side maybe assumed it would get a dummy frame or something.

[ncmncm]

For a real-time attitude-control feedback loop to have been subject to upset by such a bug is, exactly, the design flaw.

[4gotunameagain]

A quick search would have you know that the lead of the navigation for the copter is David S. Bayard, an awarded scientist in the control field in NASA.

So maybe the issue is not as simple as a news article portrays it

[ncmncm]

Then, apparently the real-time attitude-control feedback loop was delegated to somebody else less experienced. Navigation is a wholly different responsibility that may happen to rely on input from some of the same hardware.

On the other hand, scientists are often not as good at design as engineers. They are not routinely trained in it, and are expected to deduce it from first principles. (If you have seen code written by scientists, you will know what I mean.) When they err, it is typically by assuming that theory and practice are the same.

[avs733]

do you have ANY evidence to support this claim?

or are you relying on a third party article's interpretation and then extrapolating that even further?

[stevenhuang]

If the failure mode is being described accurately then it's clear to those who worked on hardware that these are not the sort of problems that should be encountered this late into the cycle (on Mars). Perhaps we just need more info on the situation. But as it currently stands, things smell a bit amateurish.

See also: https://news.ycombinator.com/item?id=27555761

[ncmncm]

The flight instability is, all by itself, ironclad proof of a design flaw. When a constant time-offset navigational input can affect real-time attitude control, that is a rookie booboo.

The exact nature of the design flaw is subject to further investigation.