Abstract:

We present a new method for joint automatic extrinsic calibration and sensor fusion for a multimodal sensor system comprising a LIDAR and an optical camera. Our approach exploits the natural alignment of depth and intensity edges when the calibration parameters are correct. Thus, in contrast to a number of existing approaches, we do not require the presence or identification of known alignment targets. On the other hand, the characteristics of each sensor modality, such as sampling pattern and information measured, are significantly different, making direct edge alignment difficult. To overcome this difficulty, we jointly fuse the data and estimate the calibration parameters. In particular, the joint processing evaluates and optimizes both the quality of edge alignment and the performance of the fusion algorithm using a common cost function on the output. We demonstrate accurate calibration in practical configurations in which depth measurements are sparse and contain no reflectivity information. Experiments on synthetic and real data obtained with a three-dimensional LIDAR sensor demonstrate the effectiveness of our approach.