Johannes L. Schönberger

About

Welcome! I am Principal Scientist at the Microsoft Mixed Reality & AI Lab in Zürich. Prior to joining Microsoft, I obtained my PhD in Computer Science as part of the Computer Vision and Geometry Group at ETH Zürich, where I was advised by Marc Pollefeys and Jan-Michael Frahm. Before, I received my MSc from UNC Chapel Hill and my BSc from TU Munich.

Research

My main research interest lies in image-based mapping and localization. More broadly, I am interested in computer vision, geometry, structure-from-motion, (multi-view) stereo, optimization, machine learning, and image processing. You can find a complete and up-to-date list of my publications on Google Scholar or further below.

Open Source

As part of my research, I developed the open-source software COLMAP - an end-to-end, general-purpose image-based 3D reconstruction software that is widely used in academia and industry. Over the years, I also contributed to a number of other scientific open source projects, including scikit-image, scikit-learn, NumPy, SciPy, etc. Lately, I don't have as much time as I would like to spend on open source software anymore, but you can find most of my activities on GitHub.

Honors & Awards

• PAMI Mark Everingham Prize (2020) - For the COLMAP SFM and MVS software library. The Everingham Prize is awared for a selfless contribution of significant benefit to other members of the computer vision community.
• ETH Medal (2019) - Awarded yearly for outstanding doctoral theses at ETH Zurich.
• Best Paper Honorable Mention (DAGM) (2015) - Awarded for the paper "Efficient Two-View Geometry Classification".

Publications

You can find my publications also on Google Scholar and ResearchGate.

2023

• ALSTER: A Local Spatio-Temporal Expert for Online 3D Semantic Reconstruction Silvan Weder, Francis Engelmann, Johannes L. Schönberger, Akihito Seki, Marc Pollefeys, Martin Oswald arxiv

  [ Paper | Bibtex ]

• Privacy Preserving Localization via Coordinate Permutations Linfei Pan, Johannes L. Schönberger, Viktor Larsson, Marc Pollefeys International Conference on Computer Vision (ICCV)

  [ Paper | Supplementary | Bibtex ]

2022

• LaMAR: Benchmarking Localization and Mapping for Augmented Reality Paul-Edouard Sarlin, Mihai Dusmanu, Johannes L. Schönberger, Pablo Speciale, Lukas Gruber, Viktor Larsson, Ondrej Miksik, Marc Pollefeys European Conference on Computer Vision (ECCV)

  [ Paper | Bibtex | Website ]

• Privacy Preserving Partial Localization Marcel Geppert, Viktor Larsson, Johannes L. Schönberger, Marc Pollefeys Conference on Computer Vision and Pattern Recognition (CVPR)

  [ Paper | Bibtex | Presentation ]

• Panoptic Multi-TSDFs: a Flexible Representation for Online Multi-resolution Volumetric Mapping and Long-term Dynamic Scene Consistency Lukas Schmid, Jeffrey Delmerico, Johannes L. Schönberger, Juan Nieto, Marc Pollefeys, Roland Siegwart, Cesar Cadena International Conference on Robotics and Automation (ICRA)

  [ Paper | Bibtex | Code ]

2021

• Reconstructing and grounding narrated instructional videos in 3D Dimitri Zhukov, Ignacio Rocco, Ivan Laptev, Josef Sivic, Johannes L. Schönberger, Bugra Tekin, Marc Pollefeys arXiv

  [ Paper ]

• Cross-Descriptor Visual Localization and Mapping Mihai Dusmanu, Ondrej Miksik, Johannes L. Schönberger, Marc Pollefeys International Conference on Computer Vision (ICCV)

  [ Paper | Bibtex ]

• Privacy Preserving Localization and Mapping from Uncalibrated Cameras Marcel Geppert, Viktor Larsson, Pablo Speciale Johannes L. Schönberger, Marc Pollefeys Conference on Computer Vision and Pattern Recognition (CVPR)

  [ Paper | Bibtex ]

• Privacy-Preserving Image Features via Adversarial Affine Subspace Embeddings Mihai Dusmanu, Johannes L. Schönberger, Sudipta N. Sinha, Marc Pollefeys Conference on Computer Vision and Pattern Recognition (CVPR)

  [ Paper | Supplementary | Bibtex | Project ]

• NeuralFusion: Online Depth Fusion in Latent Space Silvan Weder, Johannes L. Schönberger, Marc Pollefeys, Martin R. Oswald Conference on Computer Vision and Pattern Recognition (CVPR)

  [ Paper | Bibtex ]

2020

• Multi-View Optimization of Local Feature Geometry Mihai Dusmanu, Johannes L. Schönberger, Marc Pollefeys European Conference on Computer Vision (ECCV)

  [ Paper | Bibtex | Project | Code ]

• Privacy Preserving Structure-from-Motion Marcel Geppert, Viktor Larsson, Pablo Speciale Johannes L. Schönberger, Marc Pollefeys European Conference on Computer Vision (ECCV)

  [ Paper | Bibtex | Code ]

• RoutedFusion: Learning Real-time Depth Map Fusion Silvan Weder, Johannes L. Schönberger, Marc Pollefeys, Martin R. Oswald Conference on Computer Vision and Pattern Recognition (CVPR)

  [ Paper | Bibtex | Code ]

• HoloLens 2 Research Mode as a Tool for Computer Vision Research Dorin Ungureanu, Federica Bogo, Silvano Galliani, Pooja Sama, Xin Duan, Casey Meekhof, Jan Stühmer, Thomas J. Cashman, Bugra Tekin, Johannes L. Schönberger, Pawel Olszta, Marc Pollefeys arXiv

  [ Paper | Bibtex | Code ]

2019

• Privacy Preserving Image Queries for Camera Localization Pablo Speciale, Johannes L. Schönberger, Sudipta N. Sinha, Marc Pollefeys International Conference on Computer Vision (ICCV)

  [ Paper | Bibtex ]

• Privacy Preserving Image-Based Localization Pablo Speciale, Johannes L. Schönberger, Sing Bing Kang, Sudipta N. Sinha, Marc Pollefeys Conference on Computer Vision and Pattern Recognition (CVPR)

  [ Paper | Bibtex ]

• SciPy 1.0 - Fundamental Algorithms for Scientific Computing in Python Virtanen, Gommers, Oliphant, SciPy 1.0 Contributors Nature Methods

  [ Paper | Bibtex | Code ]

2018

• Robust Methods for Accurate and Efficient 3D Modeling from Unstructured Imagery Johannes L. Schönberger PhD Thesis

  [ Thesis | Bibtex ]

• Learning to Fuse Proposals from Multiple Scanline Optimizations in Semi-Global Matching Johannes L. Schönberger, Sudipta N. Sinha, Marc Pollefeys European Conference on Computer Vision (ECCV)

  [ Paper | Supplementary | Bibtex ]

• Learning Priors for Semantic 3D Reconstruction Ian Cherabier, Johannes L. Schönberger, Martin Oswald, Marc Pollefeys, Andreas Geiger European Conference on Computer Vision (ECCV)

  [ Paper | Supplementary | Bibtex ]

• VSO: Visual Semantic Odometry Nektarios Lianos, Johannes L. Schönberger, Marc Pollefeys, Torsten Sattler European Conference on Computer Vision (ECCV)

  [ Paper | Supplementary | Bibtex ]

• Semantic Visual Localization Johannes L. Schönberger, Marc Pollefeys, Andreas Geiger, Torsten Sattler Conference on Computer Vision and Pattern Recognition (CVPR)

  [ Paper | Supplementary | Bibtex ]

• Augmenting Crowd-Sourced 3D Reconstructions using Semantic Detections True Price, Johannes L. Schönberger, Zhen Wei, Marc Pollefeys, Jan-Michael Frahm Conference on Computer Vision and Pattern Recognition (CVPR)

  [ Paper | Supplementary | Bibtex ]

2017

• Comparative Evaluation of Hand-Crafted and Learned Local Features Johannes L. Schönberger, Hans Hardmeier, Torsten Sattler, Marc Pollefeys Conference on Computer Vision and Pattern Recognition (CVPR)

  [ Benchmark | Paper | Supplementary | Bibtex ]

• A Multi-View Stereo Benchmark with High-Resolution Images and Multi-Camera Videos Thomas Schöps, Johannes L. Schönberger, Silvano Galliani, Torsten Sattler, Konrad Schindler, Marc Pollefeys, Andreas Geiger Conference on Computer Vision and Pattern Recognition (CVPR)

  [ Benchmark | Paper | Supplementary | Bibtex ]

• Mapping on the Fly: Real-Time 3D Dense Reconstruction, Digital Surface Map and Incremental Orthomosaic Generation for Unmanned Aerial Vehicles Timo Hinzmann, Johannes L. Schönberger, Marc Pollefeys, Roland Siegwart International Conference on Field and Service Robotics (FSR)

  [ Code | Paper | Supplementary | Bibtex ]

2016

• A Vote­-and­-Verify Strategy for Fast Spatial Verification in Image Retrieval Johannes L. Schönberger, True Price, Torsten Sattler, Jan-Michael Frahm, Marc Pollefeys Asian Conference on Computer Vision (ACCV)

  [ Code | Paper | Supplementary | Bibtex ]

• Pixelwise View Selection for Unstructured Multi-View Stereo Johannes L. Schönberger, Enliang Zheng, Marc Pollefeys, Jan-Michael Frahm European Conference on Computer Vision (ECCV)

  [ Paper | Supplementary | Bibtex | Video ]

• Indoor-Outdoor 3D Reconstruction Alignment Andrea Cohen, Johannes L. Schönberger, Pablo Speciale, Torsten Sattler, Jan-Michael Frahm, Marc Pollefeys European Conference on Computer Vision (ECCV)

  [ Paper | Supplementary | Bibtex | Video ]

• Structure-from-Motion Revisited Johannes L. Schönberger, Jan-Michael Frahm Conference on Computer Vision and Pattern Recognition (CVPR)

  [ Website | Paper | Supplementary | Bibtex | Video ]

• From Dusk till Dawn: Modeling in the Dark Filip Radenovic, Johannes L. Schönberger, Dinhuang Ji, Jan-Michael Frahm, Ondrej Chum, Jiri Matas Conference on Computer Vision and Pattern Recognition (CVPR)

  [ Paper | Supplementary | Bibtex | Video ]

2015

• Efficient Two-View Geometry Classification Johannes L. Schönberger, Alexander C. Berg, Jan-Michael Frahm German Conference on Pattern Recognition (GCPR) Best Paper Honorable Mention

  [ Paper | Bibtex ]

• PAIGE: PAirwise Image Geometry Encoding for Improved Efficiency in Structure-from-Motion Johannes L. Schönberger, Alexander C. Berg, Jan-Michael Frahm Conference on Computer Vision and Pattern Recognition (CVPR)

  [ Paper | Bibtex ]

• From Single Image Query to Detailed 3D Reconstruction Johannes L. Schönberger, Filip Radenovic, Ondrej Chum, Jan-Michael Frahm Conference on Computer Vision and Pattern Recognition (CVPR)

  [ Paper | Bibtex | Video ]

• Reconstructing the World* in Six Days *(As Captured by the Yahoo 100 Million Image Dataset) Jared Heinly, Johannes L. Schönberger, Enrique Dunn, Jan-Michael Frahm Conference on Computer Vision and Pattern Recognition (CVPR)

  [ Website | Paper | Bibtex ]

2014

• scikit-image: Image processing in Python Stéfan van der Walt, Johannes L. Schönberger, Juan Nunez-Iglesias, François Boulogne, Joshua D. Warner, Neil Yager, Emmanuelle Gouillart, Tony Yu, scikit-image contributors PeerJ

  [ Website | Paper | Bibtex ]

• Structure-from-motion for MAV image sequence analysis with photogrammetric applications Johannes L. Schönberger, Friedrich Fraundorfer, Jan-Michael Frahm Photogrammetric Computer Vision Symposium (PCV)

  [ Code | Paper | Bibtex ]

• Benefit of Using Multiple Baselines and Multiple Aspects for SAR Interferometry of Urban Areas Michael Schmitt, Johannes L. Schönberger, Uwe Stilla IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing (JSTARS)

  [ Paper | Bibtex ]

• Adaptive Covariance Matrix Estimation for Multi-Baseline InSAR Data Stacks Michael Schmitt, Johannes L. Schönberger, Uwe Stilla IEEE Transactions on Geoscience and Remote Sensing (TGARS)

  [ Paper | Bibtex ]

2013

• Potential of Airborne Single-Pass Millimeterwave InSAR Data for Individual Tree Recognition Michael Schmitt, Alex Brück, Johannes L. Schönberger, Uwe Stilla 33. Wissenschaftlich Technische Jahrestagung der DGPF

  [ Paper | Bibtex ]

Tutorials and Workshops

• Localization and Mapping for Augmented Reality Paul-Edouard Sarlin, Mihai Dusmanu, Johannes L. Schönberger, Viktor Larsson, Ondrej Miksik, Marc Pollefeys European Conference on Computer Vision (ICCV) 2022

  [ Website ]

• Image Matching: Local Features & Beyond Vassileios Balntas, Vincent Lepetit, Johannes L. Schönberger, Eduard Trulls, Kwand Moo Yi Conference on Computer Vision and Pattern Recognition (CVPR) 2019 Conference on Computer Vision and Pattern Recognition (CVPR) 2020

  [ Website ]

• Long-Term Visual Localization Vassileios Balntas, Lars Hammarstrand, Huub Heijnen, Fredrik Kahl, Will Maddern, Krystian Mikolajczyk, Tomas Pajdla, Marc Pollefeys, Torsten Sattler, Johannes L. Schönberger, Pablo Speciale, Josef Sivic, Carl Toft, Akihiko Torii Conference on Computer Vision and Pattern Recognition (CVPR) 2019 European Conference on Computer Vision (ECCV) 2020

  [ Website (2019) | Website (2020) ]

• Microsoft HoloLens 2 and Azure Kinect DK as Tools for Computer Vision Research Marc Pollefeys, Jamie Shotton, Silvano Galliani, Federica Bogo, Johannes L. Schönberger, Dorin Ungureanu Pooja Sama European Conference on Computer Vision (ECCV) 2020

  [ Website ]

• Computer Vision Applications for Mixed Reality Headsets Marc Pollefeys, Federica Bogo, Johannes L. Schönberger, Osman Ulusoy Conference on Computer Vision and Pattern Recognition (CVPR) 2019

  [ Website ]

• HoloLens as a Tool for Computer Vision Research Marc Pollefeys, Shivkumar Swaminathan, Johannes L. Schönberger, Andrew Fitzgibbon European Conference on Computer Vision (ECCV) 2018

  [ Website ]

• COLMAP Structure-from-Motion and Multi-View Stereo Software

  [ Website ]

• Large-­scale 3D Modeling Jan-Michael Frahm, Enrique Dunn, Marc Pollefeys, Jared Heinly, Johannes L. Schönberger Conference on Computer Vision and Pattern Recognition (CVPR) 2017

  [ Website ]

• Large-­scale 3D Modeling from Crowdsourced Data Jan-Michael Frahm, Enrique Dunn, Jared Heinly, Johannes L. Schönberger International Conference on 3D Vision (3DV) 2016

  [ Website ]