Skip to main navigation Skip to search Skip to main content

OpenUrban3D: Label-Free Open-Vocabulary Semantic Segmentation of Large-Scale Urban Point Clouds

  • Xi'an Jiaotong University

Research output: Contribution to journal › Article › peer-review

4 Scopus citations

Abstract

Open-vocabulary (OV) semantic segmentation enables models to recognize and segment objects from arbitrary natural language descriptions, offering the flexibility to handle novel, fine-grained, or functionally defined categories beyond fixed label sets. While this capability is crucial for large-scale urban point clouds that support applications such as digital twins, smart city management, and urban analytics, it remains largely unexplored in this domain. The main obstacles are the frequent absence of high-quality, well-aligned multiview imagery in large-scale urban point-cloud datasets and the poor generalization of existing 3-D segmentation pipelines across diverse urban environments with substantial variation in geometry, scale, and appearance. To address these challenges, we present OpenUrban3D, a 3-D OV semantic segmentation framework for large-scale urban scenes that operates without aligned multiview images, pretrained point cloud segmentation networks, or manual labels. Our approach generates robust semantic features directly from raw point clouds through multiview, multigranularity rendering, mask-level vision–language feature extraction, and sample-balanced fusion, followed by distillation into a 3-D backbone model. This design enables zero-shot segmentation for arbitrary text queries while capturing both semantic richness and geometric priors. Extensive experiments on large-scale urban benchmarks, including SensatUrban and SUM, show that OpenUrban3D achieves significant improvements in both segmentation accuracy and cross-scene generalization over existing methods, demonstrating its potential as a flexible and scalable solution for 3-D urban scene understanding.

Original languageEnglish
Article number4501917
JournalIEEE Transactions on Geoscience and Remote Sensing
Volume64
DOIs
StatePublished - 2026

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

  1. SDG 11 - Sustainable Cities and Communities
    SDG 11 Sustainable Cities and Communities

Keywords

  • Knowledge distillation
  • label-free
  • large-scale urban point clouds
  • multiview projection
  • open-vocabulary (OV) semantic segmentation
  • vision–language models (VLMs)

Fingerprint

Dive into the research topics of 'OpenUrban3D: Label-Free Open-Vocabulary Semantic Segmentation of Large-Scale Urban Point Clouds'. Together they form a unique fingerprint.

Cite this