UniT: Unified Geometry Learning with Group Autoregressive Transformer

H. Wang, Y. Huang, Z. Kuang, H. Lu, X. Zheng. M. Yang, and G. Hua

UniT is a unified feed-forward model that reformulates a wide range of geometry perception capabilities into a single framework, covering diverse view configurations, modality combinations, metric-scale perception, and long-horizon scalability. It supports both online and offline inference over an arbitrary number of views, flexibly incorporates auxiliary modalities such as camera parameters and depth maps, recovers geometry in metric scale measured in meters, and maintains bounded complexity over long horizons in in-the-wild environments. Project Page

PacGDC: Label-Efficient Generalizable Depth Completion with Projection Ambiguity and Consistency

H. Wang, A. Xiao, X. Zhang, M. Yang, and S. Lu

PacGDC is a label-efficient technique that enhances data diversity with minimal annotation effort for generalizable depth completion. It builds on novel insights into inherent ambiguities and consistencies in object shapes and positions during 2D-to-3D projection, allowing the synthesis of numerous pseudo geometries for the same visual scene. These insights contribute to the model’s strong generalizability across diverse scene semantics/scales and depth sparsity/patterns under both zero-/few-shot settings. Project Page

Scale Propagation Network for Generalizable Depth Completion

H. Wang, M. Yang, X. Zheng, and G. Hua

SPNet is the first to carefully analyze the significance of scale propagation property in generalizable depth completion. To satisfy this property, we introduce SP-Norm, a novel normalization layer that enhances generalizability and ensures efficient convergence. Coupled with our insights for ConvNeXt V2, SPNet achieves impressive generalization across a wide range of unseen scenarios. Project Page

G2-MonoDepth: A General Framework of Generalized Depth Map Inference from Monocular RGB-X Data

H. Wang, M. Yang, N. Zheng

G2-MonoDepth investigated a novel unified task of single-view depth inference for various robots, which may be equipped with a range of sensors, such as cameras, LiDAR, ToF, or their combinations, and operate in diverse indoor and outdoor environments. Our work introduced a general and generalized framework to robustly perform in this challenging scenario by comprehensively exploring data preparation, network architecture, and loss function. Project Page

RGB-Guided Depth Map Recovery by Two-Stage Coarse-to-Fine Dense CRF Models

H. Wang, M. Yang, C. Zhu, N. Zheng

This work aims to effectively eliminate large erroneous regions in depth maps captured by low-cost depth sensors without training. It is achieved by an optimization-based approach employing the fully connected conditional random field (dense CRF) model to integrate local and global contexts from RGB-D pairs in a coarse-to-fine manner, achieving superior performance and even outperforming learning-based techniques. Project Page

Depth Map Recovery based on a Unified Depth Boundary Distortion Model

H. Wang, M. Yang, X. Lan, C. Zhu, N. Zheng

This work is the first to establish a unified model capable of identifying all kinds of distorted boundaries in depth maps, including missing, fake, and misaligned boundaries. Leveraging this unified model, we developed a filter-based pipeline to iteratively optimize low-quality depth maps, resulting in high-quality depth maps with accurate boundaries. Project Page