A Personalized Diagnostic Generation Framework Based on Multi-source Heterogeneous Data

Personalized diagnoses have not been possible due to a sear amount of data pathologists have to bear during the day-to-day routine, leading to the current generalized standards being continuously updated as new findings are reported. It is noticeable that these practical standards are developed based on multi-source heterogeneous data, including whole-slide images and pathology and clinical reports. In this study, we propose a framework that combines pathological images and medical reports to generate a personalized diagnosis result for an individual patient. We use nuclei-level image feature similarity and content-based deep learning method to search for a personalized group of populations with similar pathological characteristics, extract structured prognostic information from descriptive pathology reports of the similar patient population, and assign importance of different prognostic factors to generate a personalized pathological diagnosis result. We use multi-source heterogeneous data from TCGA (The Cancer Genome Atlas) database. The result demonstrates that our framework matches the performance of pathologists in the diagnosis of renal cell carcinoma. This framework is designed to be generic, and this could be applied to other types of cancer. The weights could provide insights into the known prognostic factors and further guide more precise clinical treatment protocols.