CytoGAN: Unpaired staining transfer by structure preservation for cytopathology image analysis

With the development of digital pathology, deep learning is increasingly being applied to endometrial cell morphology analysis for cancer screening. And cytology images with different staining may degrade the performance of these analysis algorithms. To address the impact of staining patterns, many strategies have been proposed and hematoxylin and eosin (H&E) images have been transferred to other staining styles. However, none of the existing methods are able to generate realistic cytological images with preserved cellular layout, and many important clinical structural information is lost. To address the above issues, we propose a different staining transformation model, CytoGAN, which can quickly and realistically generate images with different staining styles. It includes a novel structure preservation module that preserves the cell structure well, even if the resolution or cell size between the source and target domains do not match. Meanwhile, a stain adaptive module is designed to help the model generate realistic and high-quality endometrial cytology images. We compared our model with ten state-of-the-art stain transformation models and evaluated by two pathologists. Furthermore, in the downstream endometrial cancer classification task, our algorithm improves the robustness of the classification model on multimodal datasets, with more than 20 % improvement in accuracy. We found that generating specified specific stains from existing H&E images improves the diagnosis of endometrial cancer. Our code will be available on github.