FuPaD: Scalable Pose Estimation by Fusing Patch-Wise VGGT with Dense Bundle Adjustment

Pose estimation, a cornerstone of 3D computer vision, is crucial for applications such as autonomous driving and augmented reality. Global feed-forward methods, such as VGGT, demonstrate potential in direct scene reconstruction and pose inference. However, they are often constrained by prohibitive memory requirements when processing long sequences typical in large-scale environments. Furthermore, the accuracy of their single-pass predictions is often limited by the absence of explicit local geometric modeling or iterative refinement. To address these limitations, we introduce FuPaD, a novel hierarchical approach for scalable pose estimation. FuPaD integrates global pose priors derived from a tailored VGGT with the local refinement offered by dense bundle adjustment (DBA). First, a tracking-informed patch sampling strategy is introduced to select salient image patches from keyframes. These patches are subsequently processed by the tailored VGGT to yield globally consistent keyframe pose priors, meanwhile significantly reducing the memory footprint compared to frame-wise processing. These global keyframe poses are then integrated with dense local pose estimates from DBA within a pose graph optimization framework. Finally, a global DBA module further refines all poses. Such hierarchical fusion ensures the global consistency while benefiting from the fine-grained local refinement provided by DBA. Evaluation on benchmarks indicates that FuPaD achieves competitive pose accuracy, particularly in large-scale scenarios, while exhibiting computational and memory efficiency.