WiMob2022
Two oral presentations at WiMob2022 in Thessaloniki, Greece
Others
International Conference · 2022
Semantic Communication for Capacity-aware Remote Collaboration
Abstract
The global spread of coronavirus has sparked a considerable interest in technologies that facilitate seamless communication between users which are physically or spatially distant. Using current remote collaboration systems that utilize 3D sensing with LiDAR and depth cameras, point cloud streaming, and MR/VR devices, distant users can communicate with each other as if they did in person. However, these systems may violate users' privacy since they can share information of their entire personal space with other users. In addition, although various point cloud compression methods have been proposed, remote transmission of 3D scenes still requires significant bandwidth. This paper proposes a 3D spatial data sharing system based on the paradigm of “semantic communication”, i.e., controlling communication in the units of semantic objects. Our system understands the semantics of the scene and leverages point cloud streaming, thereby enabling users to assert fine-grained control over their privacy. Further, the system adaptively controls the size of the data frame based on network capacity and scene context. The experimental results show that the network delay can be reduced by 96%. We have also tested our system in a commercial 4G network, showing that 3-D spatial sharing with point clouds over severe networks is possible.
Research Note
The global spread of the coronavirus drew considerable interest to technologies that let people who are physically apart communicate without friction. Remote collaboration systems built on 3D sensing with LiDAR and depth cameras, point cloud streaming and MR or VR devices allow distant users to converse much as they would in person.
Two difficulties come with that approach. The first is that information about the user's entire personal space is handed over to the other side. The second is that transmitting a 3D scene still demands substantial bandwidth, despite the various point cloud compression methods that have been proposed.
This work brings in the paradigm of semantic communication. Rather than sending data as a stream of signals, it treats the content in units of semantically meaningful objects and controls communication at that granularity. Being able to choose which objects to send and which to withhold answers both difficulties through a single mechanism, since the same choice governs privacy and bandwidth alike.
The proposed system understands the semantics of the scene and streams point clouds accordingly, giving users fine-grained control over their privacy, and it adaptively controls the size of the data frame according to network capacity and scene context. Experiments reduced network delay by 96%. Testing on a commercial 4G network further showed that 3D spatial sharing with point clouds is feasible over constrained links.
Related Projects
新生活様式におけるコミュニティ形成のためのサイバーフィジカル空間共有基盤
National Institute of Information and Communications Technology (NICT) 高度通信・放送研究開発委託研究
ウイルス等感染症対策に資する情報通信技術の研究開発 課題C: アフターコロナ社会を形成するICT
