Human Segmentation for Classroom Video: Dealing with the small size overlapped and distorted human

Segmenting humans and recognizing different areas of a body from a complex scenario is a fundamental and critical step for developing technology-enhanced classroom teaching video reviewing systems. Such systems could elevate laborious video reviewing processes and assist educators to improve their teaching quality. The current state-of-The-Art instance segmentation techniques do not meet the requirements to solve problems found in classrooms, including human overlapping and occluded, and high object size variation. Thus, this paper presents an integrated method that combines a general-purpose instance segmentation with a robust Face Detection algorithm. The proposed method can detect and segment humans in the classroom environment. Human faces are also detected and matched to each human instance to enrich the required data for classroom environment analysis. The system was trained and tested on a custom annotated dataset consist of 1,000 images of students in classrooms and situations with different sizes and numbers of human. Our combined method can segment 86.46% of the instances, with 69.50% of the mean Intersection over Union (mIoU) and perform better than end-To-end Fine-Tuned Mask RCNN.