The light transport matrix (LTM) is an instrumental tool in line-of-sight (LOS) imaging, describing how light interacts with the scene and enabling applications such as relighting or separation of illumination components. We introduce a framework to estimate the LTM of non-line-of-sight (NLOS) scenarios, coupling recent virtual forward light propagation models for NLOS imaging with the LOS light transport equation. We design computational projector-camera setups, and use these virtual imaging systems to estimate the transport matrix of hidden scenes. We introduce the specific illumination functions to compute the different elements of the matrix, overcoming the challenging wide-aperture conditions of NLOS setups. Our NLOS light transport matrix allows us to (re)illuminate specific locations of a hidden scene, and separate direct, first-order indirect, and higher-order indirect illumination of complex cluttered hidden scenes, similar to existing LOS techniques.


*Erratum: In the Publisher's version, denominator of Eq. 14 should be |x_s - x_b| instead of |x_v - x_s|.


@inproceedings{marco2021NLOSvLTM, author = {Marco, Julio and Jarabo, Adrian and Nam, Ji Hyun and Liu, Xiaochun and Cosculluela, Miguel Angel and Velten, Andreas and Gutierrez, Diego}, title = {Virtual Light Transport Matrices for Non-Line-of-Sight Imaging}, booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)}, month = {October}, year = {2021}, pages = {2440-2449} }

Related Bibtex


We would like to thank the reviewers for their feedback on the manuscript, as well as Ibón Guillén for discussions at early stages of the project. This work was funded by DARPA (REVEAL HR0011-16-C-0025), the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (CHAMELEON project, grant agreement No 682080), the Ministry of Science and Innovation of Spain (project PID2019-105004GBI00), and the University of Wisconsin-Madison Office of the Vice Chancellor for Research and Graduate Education with funding from the Wisconsin Alumni Research Foundation.