跳到主要导航 跳到搜索 跳到主要内容

Composable Multimodal Semantic Communication: A Lightweight Large AI Model Approach

  • Tantan Zhao
  • , Fan Li
  • , Xinyu Huang
  • , Yiqun Liu
  • , Arumugam Nallanathan
  • Xi'an Jiaotong University
  • University of Waterloo
  • Xi'an University of Technology
  • Queen Mary University of London
  • Kyung Hee University

科研成果: 期刊稿件文章同行评审

摘要

Multimodal semantic communication is a promising paradigm for enabling immersive intelligent services in sixth-generation (6G) networks, yet existing studies lack flexible support for arbitrary modality combinations and lightweight semantic knowledge modeling suitable for edge deployment. In this paper, we investigate a lightweight large AI model (LAM)-empowered composable multimodal semantic communication (CMSC) framework. Text is adopted as a semantic bridge to unify heterogeneous modalities, while a flexible and learnable modality composition weighting mechanism enables arbitrary combinations of image, audio, and video inputs to be aggregated into a communication-oriented semantic representation. To enhance semantic robustness, a bottleneck-aware lightweight semantic knowledge base is constructed by leveraging a frozen large language model with visual prompts, where raw images or middle video frames are jointly used with textual semantics to mitigate semantic ambiguity and compensate for semantic degradation caused by wireless channel impairments. The enhanced semantics are encoded and transmitted over wireless channels, and multimodal signals are reconstructed at the receiver via diffusion-based generation. Extensive experiments on real-world datasets show that CMSC achieves a higher compression rate while maintaining comparable transmission accuracy under both arbitrary and typical multimodal combinations, demonstrating its effectiveness for flexible and lightweight multimodal semantic communication in 6G ubiquitous intelligence.

源语言英语
期刊IEEE Transactions on Communications
DOI
出版状态已接受/待刊 - 2026
已对外发布

学术指纹

探究 'Composable Multimodal Semantic Communication: A Lightweight Large AI Model Approach' 的科研主题。它们共同构成独一无二的学术指纹。

引用此