Segmenting bones from brain MRI via generative adversarial learning

Computed tomography (CT) is the standard imaging modality for capturing bony structures that are used in craniomaxillofacial (CMF) surgical planning. Unfortunately, CT emits radiation and is not a safe imaging modality, especially for infant patients. Thus, there is a clinical need of using alternative safer modalities, e.g., magnetic resonance imaging (MRI), for those patient populations. Although MRI provides good image quality for soft tissue, it lacks bony boundary information that can be used for segmentation. In this work, we introduce a novel method to automatically segment bony structures from MRI. Our method is based on a convolutional neural network composed of an image synthesis sub-network and a segmentation sub-network. The image synthesis sub-network jointly learns the cycle-consistent mappings both from MRI-to-CT and from CT-to-MRI via generative adversarial learning. Given an image from either modality, this sub-network generates the image of the other modality, resulting in a new MRI-CT pair of which the anatomical structure information are supposed to be consistent. In this way, the bone annotations (labels) from CT modality are implicitly transferred to the MRI modality to train the segmentation sub-network. We train the model in a semi-supervised manner (i.e., make use of both paired and unpaired MRI-CT data) to solve the problem with limited number of paired MRI-CT images. Moreover, a neighbor-based anchoring method and a feature-matching-based semantic consistency regularization are proposed to ameliorate the ambiguity problem of cycle-consistent cross-modality image synthesis. Experimental results demonstrate that the proposed method can effectively boost the generalizability of the segmentation sub-network. Compared with other state-of-the-art methods using only limited paired MRI-CT data, the proposed method successfully improves the segmentation performance by using both the paired and unpaired data in a semi-supervised manner.