[flyinglizard]

Why is this better than separate LIDAR and camera?

Because you're collecting NIR ambient light, your optics are wideband. Meaning that daylight would have a more pronounced negative effect on system range (easier to saturate the photocells). It's also low resolution (as most LIDARs are), and there is no color segmentation data.

In an automotive application, I can't see a justification to unify both visual and LIDAR into a single sensor, rather than having an extrinsically calibrated array of sensors. You can improve the calibration out of the data over time if you're very concerned about system stability.

It seems like a nice party trick, but the vehicle LIDAR game focuses on solid state long range units, as this will be what gets into mass production. The visual band imagers in the car are a given for many other reasons anyway.

[dllu]

First, the optics are not wideband. It is only collecting narrow band NIR light. Saturation is not a problem. The CEO of Ouster explains this in a reddit comment [0]:

> We are not sacrificing lidar performance by adding ambient imaging functionality. The lidar subsystem has a short integration time that avoids saturation, and if anything our approach outperforms other lidars.

> As proof, the example videos linked to in the article show raw unedited point cloud data with the lidar operating in extremely sunny environments with plenty of specular reflectors. You can see lens flare in the ambient imagery as any camera would exhibit, but the lidar signal and range data are unaffected. In addition, if you point a velodyne directly at the sun its false positive rate increases significantly while our sensor's FPR does not. No lidar will return the distance to the sun so the only thing that matters is FPR in this scenario.

> We've independently verified the OS-1's range performance with customers under all levels of solar exposure and I guarantee you can't get a smaller, cheaper lidar with even close to this combination of resolution and performance. If you have any doubts, download the raw pcap files from our github page and play them back yourself. We stand behind our data, our pricing, and our spec!

Second, even if you do plan on adding extra cameras, the extrinsic calibration between camera and lidar may become easier if you have good quality ambient light measurement from the lidar. For example Jesse Levinson, cofounder of Zoox, computes extrinsic calibration between camera and Velodyne lidar by assuming that depth discontinuities are correlated with visual features [1]. But obviously the correlation between 850 nm images and visible light images would be way better.

[0] https://www.reddit.com/r/SelfDrivingCars/comments/9c60pe/the...

[1] http://www.roboticsproceedings.org/rss09/p29.pdf

[flyinglizard]

>> First, the optics are not wideband. It is only collecting narrow band NIR light. Saturation is not a problem. The CEO of Ouster explains this in a reddit comment [0]:

They are probably wider band than would be required to read only the sensor self illumination.

>> Second, even if you do plan on adding extra cameras, the extrinsic calibration between camera and lidar may become easier if you have good quality ambient light measurement from the lidar. For example Jesse Levinson, cofounder of Zoox, computes extrinsic calibration between camera and Velodyne lidar by assuming that depth discontinuities are correlated with visual features [1]. But obviously the correlation between 850 nm images and visible light images would be way better.

I agree with that - but you could probably go the other way around and coorelate the LIDAR depth map with depth obtained through stereo imaging. The temporal synchronization between NIR and depth provided by this unit is nice though.

Let me phrase this differently - while the videos are cool to watch, I don't think calibration is the problem in vehicles, and nor are baseline artifacts between sensors when operating at such far ranges (whether your camera and LIDAR are perfectly aligned or translated 10cm apart, it won't matter much looking 10m down the road).

Having moving parts, however, won't get this system into a production model.

[deepnotderp]

It's better because there's no need for calibration, you always have perfect calibration.

Solid state lidar has issues. The cofounder of Ouster, Angus Pacala previously cofounded Quanergy, a solid state lidar startup.

[flyinglizard]

Solid state LIDAR certainly has issues - but someone is going to solve those and this is what will get into automotive, definitely not $10k units with moving parts.

There was an announcement on a cooperation between BMW and Innoviz (an Israeli maker of solid state LIDARs) with Magna being their OEM sponsor.

I'm not sure calibration is that big of a deal for this application. Sensors are going to be calibrated and tested in the factory or at a module level regardless, and the accuracy requirements in automotive are much lower than consumer products using similar technology.

You can't overcome not having colors (traffic lights, anyone?), limited ranging distance or sensor saturation due to ambient conditions.

[jimduk]

Agreed, spent time last year on a project fusing lidar and rotating LWIR (thermal) with some smart people and calibration took significant effort, mechanically, in electronic timing, and in algorithmic fusing. This looks like a nice step forward.