About

Welcome! I am a Principal Scientist at the Microsoft MR & AI Lab in Zürich and currently visiting CVG at ETH Zürich as an academic guest. Before, I obtained a PhD in Computer Science as part of CVG at ETH Zürich. I received my MSc from UNC Chapel Hill and my BSc from TU Munich.

Research

My main research interests lie in image-based mapping, localization, and scene understanding. More broadly, I am interested in topics at the intersection of computer vision and geometry. A complete and up-to-date list of publications can be found on Google Scholar or further below.

As part of my research, I developed the open-source software COLMAP - an end-to-end image-based 3D reconstruction software that is widely used in academia and industry. Over the years, I also contributed to a number of other scientific open source projects, including scikit-image, scikit-learn, NumPy, SciPy, etc. Most open-source activities can be found on GitHub.

Awards

I received the ECVA Young Researcher Award in 2023 and, for my work on the COLMAP SFM and MVS software, I received the PAMI Mark Everingham Prize in 2020. Further, I received an ETH Medal in 2019 for my PhD thesis as well as a best paper honorable mention at DAGM 2015.

Publications

List of papers and patents in reverse chronological order. For more information, please visit Google Scholar.

Papers

1. Global Structure-from-Motion Revisited

   Linfei Pan, Daniel Barath, Marc Pollefeys, Johannes Schönberger

   European Conference on Computer Vision (ECCV), 2024

   [ Project | Paper | Supplementary | Bibtex | Code ]

   

2. Robust Incremental Structure-from-Motion with Hybrid Features

   Shaohui Liu, Yidan Gao, Tianyi Zhang, Rémi Pautrat, Johannes Schönberger, Viktor Larsson, Marc Pollefeys

   European Conference on Computer Vision (ECCV), 2024

   [ Paper | Supplementary | Bibtex ]

   

3. A Local Spatio-Temporal Expert for Online 3D Semantic Reconstruction

   Silvan Weder, Francis Engelmann, Johannes Schönberger, Akihito Seki, Marc Pollefeys, Martin Oswald

   arXiv, 2023

   [ Paper | Bibtex ]

   

4. LaMAR: Benchmarking Localization and Mapping for Augmented Reality

   Paul-Edouard Sarlin, Mihai Dusmanu, Johannes Schönberger, Pablo Speciale, Lukas Gruber, Viktor Larsson, Ondrej Miksik, Marc Pollefeys

   European Conference on Computer Vision (ECCV), 2022

   [ Paper | Bibtex | Website | Video ]

   

5. Panoptic Multi-TSDFs: a Flexible Representation for Online Multi-resolution Volumetric Mapping and Long-term Dynamic Scene Consistency

   Lukas Schmid, Jeffrey Delmerico, Johannes Schönberger, Juan Nieto, Marc Pollefeys, Roland Siegwart, Cesar Cadena

   International Conference on Robotics and Automation (ICRA), 2022

   [ Paper | Bibtex | Code ]

   

6. Reconstructing and grounding narrated instructional videos in 3D

   Dimitri Zhukov, Ignacio Rocco, Ivan Laptev, Josef Sivic, Johannes Schönberger, Bugra Tekin, Marc Pollefeys

   arXiv, 2021

   [ Paper | Bibtex ]

   

7. Cross-Descriptor Visual Localization and Mapping

   Mihai Dusmanu, Ondrej Miksik, Johannes Schönberger, Marc Pollefeys

   International Conference on Computer Vision (ICCV), 2021

   [ Paper | Bibtex ]

   

8. Privacy Preserving Localization and Mapping from Uncalibrated Cameras

   Marcel Geppert, Viktor Larsson, Pablo Speciale Johannes Schönberger, Marc Pollefeys

   Conference on Computer Vision and Pattern Recognition (CVPR), 2021

   [ Paper | Bibtex ]

   

9. Privacy-Preserving Image Features via Adversarial Affine Subspace Embeddings

   Mihai Dusmanu, Johannes Schönberger, Sudipta N. Sinha, Marc Pollefeys

   Conference on Computer Vision and Pattern Recognition (CVPR), 2021

   [ Paper | Supplementary | Bibtex | Project ]

   

10. NeuralFusion: Online Depth Fusion in Latent Space

    Silvan Weder, Johannes Schönberger, Marc Pollefeys, Martin R. Oswald

    Conference on Computer Vision and Pattern Recognition (CVPR), 2021

    [ Paper | Bibtex ]

    

11. Multi-View Optimization of Local Feature Geometry

    Mihai Dusmanu, Johannes Schönberger, Marc Pollefeys

    European Conference on Computer Vision (ECCV), 2020

    [ Paper | Bibtex | Project | Code ]

    

12. Privacy Preserving Structure-from-Motion

    Marcel Geppert, Viktor Larsson, Pablo Speciale Johannes Schönberger, Marc Pollefeys

    European Conference on Computer Vision (ECCV), 2020

    [ Paper | Bibtex | Code ]

    

13. RoutedFusion: Learning Real-time Depth Map Fusion

    Silvan Weder, Johannes Schönberger, Marc Pollefeys, Martin R. Oswald

    Conference on Computer Vision and Pattern Recognition (CVPR), 2020

    [ Paper | Bibtex | Code ]

    

14. HoloLens 2 Research Mode as a Tool for Computer Vision Research

    Dorin Ungureanu, Federica Bogo, Silvano Galliani, Pooja Sama, Xin Duan, Casey Meekhof, Jan Stühmer, Thomas J. Cashman, Bugra Tekin, Johannes Schönberger, Pawel Olszta, Marc Pollefeys

    arXiv, 2020

    [ Paper | Bibtex | Code ]

    

15. Privacy Preserving Image Queries for Camera Localization

    Pablo Speciale, Johannes Schönberger, Sudipta N. Sinha, Marc Pollefeys

    International Conference on Computer Vision (ICCV), 2019

    [ Paper | Bibtex ]

    

16. Privacy Preserving Image-Based Localization

    Pablo Speciale, Johannes Schönberger, Sing Bing Kang, Sudipta N. Sinha, Marc Pollefeys

    Conference on Computer Vision and Pattern Recognition (CVPR), 2019

    [ Paper | Bibtex ]

    

17. SciPy 1.0 - Fundamental Algorithms for Scientific Computing in Python

    Virtanen, Gommers, Oliphant, SciPy 1.0 Contributors

    Nature Methods, 2019

    [ Paper | Bibtex | Code ]

    

18. Robust Methods for Accurate and Efficient 3D Modeling from Unstructured Imagery

    Johannes Schönberger

    PhD Thesis, 2018

    [ Thesis | Bibtex ]

    

19. Learning to Fuse Proposals from Multiple Scanline Optimizations in Semi-Global Matching

    Johannes Schönberger, Sudipta N. Sinha, Marc Pollefeys

    European Conference on Computer Vision (ECCV), 2018

    [ Paper | Supplementary | Bibtex ]

    

20. Learning Priors for Semantic 3D Reconstruction

    Ian Cherabier, Johannes Schönberger, Martin Oswald, Marc Pollefeys, Andreas Geiger

    European Conference on Computer Vision (ECCV), 2018

    [ Paper | Supplementary | Bibtex ]

    

21. VSO: Visual Semantic Odometry

    Nektarios Lianos, Johannes Schönberger, Marc Pollefeys, Torsten Sattler

    European Conference on Computer Vision (ECCV), 2018

    [ Paper | Supplementary | Bibtex ]

    

22. Semantic Visual Localization

    Johannes Schönberger, Marc Pollefeys, Andreas Geiger, Torsten Sattler

    Conference on Computer Vision and Pattern Recognition (CVPR), 2018

    [ Paper | Supplementary | Bibtex ]

    

23. Augmenting Crowd-Sourced 3D Reconstructions using Semantic Detections

    True Price, Johannes Schönberger, Zhen Wei, Marc Pollefeys, Jan-Michael Frahm

    Conference on Computer Vision and Pattern Recognition (CVPR), 2018

    [ Paper | Supplementary | Bibtex ]

    

24. Comparative Evaluation of Hand-Crafted and Learned Local Features

    Johannes Schönberger, Hans Hardmeier, Torsten Sattler, Marc Pollefeys

    Conference on Computer Vision and Pattern Recognition (CVPR), 2017

    [ Benchmark | Paper | Supplementary | Bibtex ]

    

25. A Multi-View Stereo Benchmark with High-Resolution Images and Multi-Camera Videos

    Thomas Schöps, Johannes Schönberger, Silvano Galliani, Torsten Sattler, Konrad Schindler, Marc Pollefeys, Andreas Geiger

    Conference on Computer Vision and Pattern Recognition (CVPR), 2017

    [ Paper | Supplementary | Bibtex ]

    

26. Mapping on the Fly: Real-Time 3D Dense Reconstruction, Digital Surface Map and Incremental Orthomosaic Generation for Unmanned Aerial Vehicles

    Timo Hinzmann, Johannes Schönberger, Marc Pollefeys, Roland Siegwart

    International Conference on Field and Service Robotics (FSR), 2017

    [ Code | Paper | Supplementary | Bibtex ]

    

27. A Vote­-and­-Verify Strategy for Fast Spatial Verification in Image Retrieval

    Johannes Schönberger, True Price, Torsten Sattler, Jan-Michael Frahm, Marc Pollefeys

    Asian Conference on Computer Vision (ACCV), 2016

    [ Code | Paper | Supplementary | Bibtex ]

    

28. Pixelwise View Selection for Unstructured Multi-View Stereo

    Johannes Schönberger, Enliang Zheng, Marc Pollefeys, Jan-Michael Frahm

    European Conference on Computer Vision (ECCV), 2016

    [ Paper | Supplementary | Bibtex | Video ]

    

29. Indoor-Outdoor 3D Reconstruction Alignment

    Andrea Cohen, Johannes Schönberger, Pablo Speciale, Torsten Sattler, Jan-Michael Frahm, Marc Pollefeys

    European Conference on Computer Vision (ECCV), 2016

    [ Paper | Supplementary | Bibtex | Video ]

    

30. Structure-from-Motion Revisited

    Johannes Schönberger, Jan-Michael Frahm

    Conference on Computer Vision and Pattern Recognition (CVPR), 2016

    [ Website | Paper | Supplementary | Bibtex | Video ]

    

31. From Dusk till Dawn: Modeling in the Dark

    Filip Radenovic, Johannes Schönberger, Dinhuang Ji, Jan-Michael Frahm, Ondrej Chum, Jiri Matas

    Conference on Computer Vision and Pattern Recognition (CVPR), 2016

    [ Paper | Supplementary | Bibtex | Video ]

    

32. Efficient Two-View Geometry Classification

    Johannes Schönberger, Alexander C. Berg, Jan-Michael Frahm

    German Conference on Pattern Recognition (GCPR), 2015

    [ Paper | Bibtex ]

    

33. PAIGE: PAirwise Image Geometry Encoding for Improved Efficiency in Structure-from-Motion

    Johannes Schönberger, Alexander C. Berg, Jan-Michael Frahm

    Conference on Computer Vision and Pattern Recognition (CVPR), 2015

    [ Paper | Bibtex ]

    

34. From Single Image Query to Detailed 3D Reconstruction

    Johannes Schönberger, Filip Radenovic, Ondrej Chum, Jan-Michael Frahm

    Conference on Computer Vision and Pattern Recognition (CVPR), 2015

    [ Paper | Bibtex | Video ]

    

35. Reconstructing the World* in Six Days *(As Captured by the Yahoo 100 Million Image Dataset)

    Jared Heinly, Johannes Schönberger, Enrique Dunn, Jan-Michael Frahm

    Conference on Computer Vision and Pattern Recognition (CVPR), 2015

    [ Website | Paper | Bibtex ]

    

36. scikit-image: Image processing in Python

    Stéfan van der Walt, Johannes Schönberger, Juan Nunez-Iglesias, François Boulogne, Joshua D. Warner, Neil Yager, Emmanuelle Gouillart, Tony Yu, scikit-image contributors

    PeerJ, 2014

    [ Website | Paper | Bibtex ]

    

37. Structure-from-motion for MAV image sequence analysis with photogrammetric applications

    Johannes Schönberger, Friedrich Fraundorfer, Jan-Michael Frahm

    Photogrammetric Computer Vision Symposium (PCV), 2014

    [ Code | Paper | Bibtex ]

    

38. Benefit of Using Multiple Baselines and Multiple Aspects for SAR Interferometry of Urban Areas

    Michael Schmitt, Johannes Schönberger, Uwe Stilla

    IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing (JSTARS), 2014

    [ Paper | Bibtex ]

    

39. Adaptive Covariance Matrix Estimation for Multi-Baseline InSAR Data Stacks

    Michael Schmitt, Johannes Schönberger, Uwe Stilla

    IEEE Transactions on Geoscience and Remote Sensing (TGARS), 2014

    [ Paper | Bibtex ]

    

40. Potential of Airborne Single-Pass Millimeterwave InSAR Data for Individual Tree Recognition

    Michael Schmitt, Alex Brück, Johannes Schönberger, Uwe Stilla

    33. Wissenschaftlich Technische Jahrestagung der DGPF, 2013

    [ Paper | Bibtex ]

    

Patents

1. Matching Between 2D And 3D For Direct Localization

   Johannes Schönberger, Rui Wang, Prune Truong, Marc Pollefeys

   US Patent App. 18/055,722, 2024

2. Access Authentication Using Obfuscated Biometrics

   Johannes Schönberger, Marc Pollefeys

   US Patent App. 17/997,081, 2023

3. Collecting Telemetry Data for 3D Map Updates

   Johannes Schönberger, Marc Pollefeys

   US Patent App. 17/751,193, 2023

4. Updating a 3D Map of an Environment

   Christoph Vogel, Jan-Willem Buurlage, Johannes Schönberger, Juan Nieto, Marc Pollefeys, Timon Knigge, Marcel Geppert

   US Patent App. 17/743,430, 2023

5. Facade Biasing for Reflection Correction in Photogrammetric Reconstruction

   Michael Jones, Johannes Schönberger

   WO 2023 023266 A1, 2023

6. Image-based Localization

   Johannes Schönberger, Marc Pollefeys

   US Patent 11,145,083, 2021

7. Fusing Disparity Proposals in Stereo Matching

   Sudipta Sinha, Marc Pollefeys, Johannes Schönberger

   US Patent 10,878,590, 2020

8. Methods, Systems, and Computer Readable Media for Image Overlap Detection

   Jared Heinly, Johannes Schönberger, Enrique Dunn, Jan-Michael Frahm

   US Patent 10,504,000, 2018

Miscellaneous

Selected Talks, Workshops, Tutorials

1. ECCV 2022, Co-organizer, “Tutorial on Localization and Mapping for Augmented Reality”
2. CVPR 2020/2021/2022, Co-organizer, “Image Matching: Local Features & Beyond”
3. CVPR2019/2020, ECCV2020, Co-organizer, “Long-Term Visual Localization Benchmark”
4. ECCV 2020, Co-organizer, Los Angeles, “Microsoft HoloLens 2 and Azure Kinect DK as Tools for Computer Vision Research”
5. CVPR 2019, 3D Scene Generation, Speaker, Los Angeles, “3D Scene Reconstruction from Unstructured Imagery”
6. CVPR 2019, Co-organizer, Los Angeles, “Computer Vision Applications for Mixed Reality Headsets”
7. CVPR 2018, Robust Vision Challenge, Co-organizer and Speaker, Salt Lake City, “Robust and Accurate Image-Based 3D Modeling”
8. CVPR 2018, Co-organizer, Los Angeles, “HoloLens as a Tool for Computer Vision Research”
9. Microsoft Research AI 2018, Speaker, Redmond, “Efficient and Accurate Image-Based 3D Modeling”
10. CVPR 2017, Co-organizer and Speaker, Hawaii, “Tutorial: Large-scale 3D Modeling from Crowdsourced Data”
11. 3DV 2016, Co-organizer and Speaker, Stanford University, “Tutorial: Large-scale 3D Modeling from Crowdsourced Data”
12. GCPR 2015, Speaker, RWTH Aachen, “Efficient Two-View Geometry Classification”
13. CMP 2015, Speaker, CVUT Prague, “PAIGE: PAirwise Image Geometry Encoding for Improved Efficiency in Structure-from-Motion”