Dissertation Defense: Jiazhang Wang, "Computational 3D Imaging of Specular and Shiny Objects"

Title

Computational 3D Imaging of Specular and Shiny Objects

Abstract:

Accurate, robust, and fast 3D reconstruction of specular surfaces holds considerable significance in various applications such as industrial inspection, cultural heritage analysis, or eye tracking. However, it is a challenging task as most established computational 3D imaging techniques like stereo triangulation or time-of-flight are primarily effective only on diffuse surfaces.

This dissertation introduces novel techniques for the measurement of specular surfaces exploiting Deflectometric information. 

First, a novel eye tracking method employing Deflectometry is introduced. Deflectometry is an optical metrology method for specular surface reconstruction. Leveraging the dense 3D information of the eye surface obtained from Deflectometry, we demonstrate experimental results for gaze error estimation that outperform current state-of-the-art methods. To enhance the robustness of our approach across diverse application scenarios, we have developed two additional Deflectometric eye tracking methods which utilize differentiable rendering and deep learning. In addition to exploring this novel application field for Deflectometry, this dissertation addresses two inherent limitations of Deflectometry for potential broader application. Firstly, Deflectometry's limited coverage, attributed to the illuminators’ size and objects’ shape, is mitigated by employing event-based triangulation in conjunction with Deflectometry. This new approach enables the use of all diffuse background elements as illumination sources, enabling reconstruction of scenes containing both diffuse and specular objects. Secondly, the normal-height ambiguity issue with single-camera Deflectometry is addressed through the development of a polarization-guided Deflectometry method, ensuring accurate and unambiguous reconstruction. This method also enhances the performance of shape-from-polarization, a prominent technique in the computer vision community for reconstructing specular surfaces.