Point-Plane SLAM for RGB-D Sensors and Its Applications

Real-time 3D reconstruction using an RGB-D sensor on a tablet.

MERL Researchers: (Computer Vision).

Search MERL publications by keyword: Computer Vision, SLAM, 3D reconstruction, registration, plane extraction,

We present a real-time 3D reconstruction system using an RGB-D sensor on a hand-held tablet. The main novelty of the system is a simultaneous localization and mapping (SLAM) algorithm that uses both point and plane features as primitives. Planes are the most common structures in man-made indoor and outdoor scenes.

In contrast to existing SLAM algorithms that use only point features, our algorithm that also exploits plane features has the following advantages:

  • (1) it enables faster correspondence search and registration, since the number of planes is typically much smaller than the number of points;
  • (2) planes generated by many points are more robust to noise than individual points, leading to more accurate registration; and
  • (3) it produces plane-based 3D models that are more compact than point-based ones.

As the core of the algorithm, we show that it is possible to register 3D data in two different coordinate systems using any combination of three point/plane features (3 planes, 2 planes and 1 point, 1 plane and 2 points, and 3 points). We use the minimal set of features in a RANSAC framework to robustly compute correspondences and estimate the camera pose. We also use the point and plane features in bundle adjustment to jointly refine all the camera poses and point/plane parameters.

Our system enables large-scale indoor 3D reconstruction and generates plane-based and point-based 3D models. The 3D models can be used for several applications, including robot collision avoidance (demonstrated at iREX 2013 and MODEX 2014), 2D camera localization for augmented reality (demonstrated at 3DV 2014 and MVA 2015), and calibration of non-overlapping cameras (presented at 3DV 2014).


Videos



Software & Data Downloads



MERL News & Events



MERL Publications

  •  Taguchi, Y., Jian, Y.-D., Ramalingam, S., Feng, C., "SLAM Using Both Points and Planes for Hand-Held 3D Sensors", International Symposium on Mixed and Augmented Reality (ISMAR), November 2012.
    BibTeX TR2012-091 PDF Software
    • @inproceedings{Taguchi2012nov,
    • author = {Taguchi, Y. and Jian, Y.-D. and Ramalingam, S. and Feng, C.},
    • title = {SLAM Using Both Points and Planes for Hand-Held 3D Sensors},
    • booktitle = {International Symposium on Mixed and Augmented Reality (ISMAR)},
    • year = 2012,
    • month = nov,
    • url = {https://www.merl.com/publications/TR2012-091}
    • }
  •  Taguchi, Y., Jian, Y.-D., Ramalingam, S., Feng, C., "Point-Plane SLAM for Hand-Held 3D Sensors", IEEE International Conference on Robotics and Automation (ICRA), DOI: 10.1109/​ICRA.2013.6631318, May 2013, pp. 5182-5189.
    BibTeX TR2013-031 PDF Software
    • @inproceedings{Taguchi2013may,
    • author = {Taguchi, Y. and Jian, Y.-D. and Ramalingam, S. and Feng, C.},
    • title = {Point-Plane SLAM for Hand-Held 3D Sensors},
    • booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
    • year = 2013,
    • pages = {5182--5189},
    • month = may,
    • doi = {10.1109/ICRA.2013.6631318},
    • issn = {1050-4729},
    • isbn = {978-1-4673-5641-1},
    • url = {https://www.merl.com/publications/TR2013-031}
    • }
  •  Ataer-Cansizoglu, E., Taguchi, Y., Ramalingam, S., Garaas, T., "Tracking an RGB-D Camera Using Points and Planes", IEEE Workshop on Consumer Depth Cameras for Computer Vision (CDC4CV), December 2013.
    BibTeX TR2013-106 PDF Software
    • @inproceedings{Ataer-Cansizoglu2013dec,
    • author = {Ataer-Cansizoglu, E. and Taguchi, Y. and Ramalingam, S. and Garaas, T.},
    • title = {Tracking an RGB-D Camera Using Points and Planes},
    • booktitle = {IEEE Workshop on Consumer Depth Cameras for Computer Vision (CDC4CV)},
    • year = 2013,
    • month = dec,
    • url = {https://www.merl.com/publications/TR2013-106}
    • }
  •  Feng, C., Taguchi, Y., Kamat, V., "Fast Plane Extraction in Organized Point Clouds Using Agglomerative Hierarchical Clustering", IEEE International Conference on Robotics and Automation (ICRA), DOI: 10.1109/​ICRA.2014.6907776, May 2014, pp. 6218-6225.
    BibTeX TR2014-066 PDF Software
    • @inproceedings{Feng2014may,
    • author = {Feng, C. and Taguchi, Y. and Kamat, V.},
    • title = {Fast Plane Extraction in Organized Point Clouds Using Agglomerative Hierarchical Clustering},
    • booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
    • year = 2014,
    • pages = {6218--6225},
    • month = may,
    • publisher = {IEEE},
    • doi = {10.1109/ICRA.2014.6907776},
    • url = {https://www.merl.com/publications/TR2014-066}
    • }
  •  Ataer-Cansizoglu, E., Taguchi, Y., Ramalingam, S., Miki, Y., "Calibration of Non-overlapping Cameras Using an External SLAM System", International Conference on 3D Vision (3DV), DOI: 10*1109/​eDV.2014.106, December 2014, pp. 509-516.
    BibTeX TR2014-106 PDF Software
    • @inproceedings{Cansizoglu2014dec,
    • author = {Ataer-Cansizoglu, E. and Taguchi, Y. and Ramalingam, S. and Miki, Y.},
    • title = {Calibration of Non-overlapping Cameras Using an External SLAM System},
    • booktitle = {International Conference on 3D Vision (3DV)},
    • year = 2014,
    • pages = {509--516},
    • month = dec,
    • publisher = {IEEE},
    • doi = {10*1109/eDV.2014.106},
    • url = {https://www.merl.com/publications/TR2014-106}
    • }