A Triplet network framework based automatic assessment of simulation quality for respiratory droplet propagation

Respiratory droplet propagation has been extensively explored with simulation and experimental methods. However, there still exists a huge gap between these methods, making automatic assessment of simulation quality quantitatively being a challenge. To address above problem, in this work, a triplet neural network framework with multi-scale CNN-BiLSTM network is developed. Firstly, Conditional Variational Auto-Encoder (CVAE) is utilized to generate multi-view simulations. Secondly, YOLOv3 is adopted to extract droplet regions of real image and simulation results. Then, a multi-scale CNN-BiLSTM network with attentive temporal pooling is designed to extract and aggregate temporal information across consecutive frames. Finally, all above networks are constructed into a triplet structure with triplet loss, and a regularization constraint being denoted as reconstruction term and prediction term is proposed. To demonstrate the performance of our approach, a new dataset is established including real sequences of cough droplets and simulation results. We validate the effectiveness and feasibility of our proposed framework using our dataset and two benchmarks, the PSB dataset and the ETH dataset, for 3D object retrieval. Our approach outperforms state-of-the-arts on our dataset and achieves comparative performance on PSB and ETH for 3D object retrieval, given quantitative quality assessment of simulation for droplet respiratory propagation automatically.