Large-­scale 3D Modeling

Description

Large-scale image-based 3D modeling has been a major goal of computer vision, enabling a wide range of applications including virtual reality, image-based localization, and autonomous navigation. One of the most diverse data sources for modeling is Internet photo collections. In the last decade, the computer vision community has made tremendous progress in large-scale structure-from-motion and multi-view stereo from Internet datasets. However, utilizing this wealth of information for 3D modeling remains a challenging problem due to the ever-increasing amount of image data. In a short period of time, research in large-scale modeling has progressed from modeling using several thousand images, to modeling from city-scale datasets of several million, and recently to reconstructing an Internet-scale dataset comprising 100 million images. This tutorial will present the main underlying technologies enabling these innovations.

Schedule (July 21, 2017 9:00-17:30)

• 9:00-11:15
  (10:00-10:45 break)
  Motivation and Basic Principles (Slides 1, Slides 2)
  • Local Features
  • Feature Matching
  • Projective Geometry
  • Epipolar Geometry
  • Absolute Pose Problem
  • Robust Estimation
  • Model Selection
• 11:15-13:45
  (12:00-13:30 break)
  Sparse Modeling (Slides)
  • Structure-from-motion
  • Bundle adjustment
• 13:45-15:00
  Dense modeling (Slides)
  • Multi-View Stereo
  • Depth and Normal Estimation
  • Fusion and Surface Generation
• 15:00-17:00
  (15:30-16:15 break)
  Large-Scale Modeling Pipeline (Slides)
  • Challenges and Opportunities
    of Crowd-sourced Data
  • Image Indexing
  • Connected-Component Discovery
  • Reconstruction of Connected Components
  • Design of a Real Pipeline
• 17:00-17:30
  Resources (Slides)

Organizers

Jan-Michael Frahm
jmf (at) cs.unc.edu
Associate Professor, UNC Chapel Hill

Enrique Dunn
edunn (at) stevens.edu
Associate Professor, Stevens Institute of Technology

Marc Pollefeys
marc.pollefeys (at) inf.ethz.ch
Full Professor, ETH Zürich
Director of Science, Microsoft HoloLens

Jared Heinly
jared.heinly (at) urcventures.com
Senior Research Engineer, URC Ventures

Johannes L. Schönberger
jsch (at) inf.ethz.ch
Ph.D. Student, ETH Zürich

Links

• Streaming Component Discovery
• COLMAP (SFM and MVS software)

References

• Johannes L. Schönberger, Jan-Michael Frahm. "Structure-from-Motion Revisited", CVPR 2016.
• Jared Heinly, Johannes L. Schönberger, Enrique Dunn, Jan-Michael Frahm. "Reconstructing the World* in Six Days *(As Captured by the Yahoo 100 Million Image Dataset)", CVPR 2015.
• Johannes L Schönberger, Filip Radenovi, Ondrej Chum, Jan-Michael Frahm. "From Single Image Query to Detailed 3D Reconstruction", CVPR 2015.
• Jan-Michael Frahm, Pierre Georgel, David Gallup, Tim Johnson, Rahul Raguram, Changchang Wu, Yi-Hung Jen, Enrique Dunn, Brian Clipp, Svetlana Lazebnik, Marc Pollefeys. "Building Rome on a Cloudless Day", ECCV 2010.
• Jan-Michael Frahm, Marc Pollefeys, Svetlana Lazebnik, Christopher Zach, David Gallup, Brian Clipp, Rahul Raguram, Changchang Wu, Tim Johnson. "Fast Robust Large-scale Mapping from Video and Internet Photo Collections", In special issue "100 years of ISPRS" of the ISPRS Journal of Photogrammetry and Remote Sensing.
• Enliang Zheng, Enrique Dunn, Valdimir Jojic, Jan-Michael Frahm. "PatchMatch Based Joint View Selection and Depthmap Estimation", CVPR 2014.
• David Gallup, Marc Pollefeys, Jan-Michael Frahm. "3D Reconstruction using an n-LayerHeightmap", DAGM 2010.
• Johannes L. Schönberger, Enliang Zheng, Marc Pollefeys, Jan-Michael Frahm. "Pixelwise View Selection for Unstructured Multi-View Stereo", ECCV 2016.