Necrosis ml an encoder decoder approach for semantic segmentation and quantization of tumor necrosis using histopathology images

Abstract

Clinical pathology is one of the finest diagnostic techniques for all types of cancer, and its outcome determines the treatment plan for a patient. In this research, two types of cancers are taken into account, Osteosarcoma and Renal Cell Carcinoma, where one of the major prognostic factors is the amount of tumor necrosis created due to Neo- adjuvant chemotherapy. Osteosarcoma is a high-grade malignant bone tumor and Re- nal Cell Carcinoma is the most common type of Kidney Cancer. Tumor necrosis is a condition where the tumorous tissues cannot perform their normal metabolic func- tions and gradually die. Neo-adjuvant chemotherapy is the treatment given to a pa- tient’s body before starting the main treatment. The proposed study aims to develop an automated tool for quantitative image analysis of digital histopathology images of post-neoadjuvant resection specimens. Post-neoadjuvant chemotherapy resection spec- imens refer to the tissue samples collected from cancer patients who have undergone Neo-adjuvant chemotherapy. Even though so many tools are available for cancer di- agnosis, there is no specific tool to calculate the percentage of necrosis available in post-neoadjuvant chemotherapy resection specimens of Osteosarcoma and Renal Cell Carcinoma. Necrosis-ML is an algorithm that has been developed in this work to perform image- level segmentation and quantization. The two major tasks involved in this tool are the semantic segmentation and quantization of segmented masks. In this algorithm, the in- put image will undergo a patchification process and segmentation will be performed at patch-level. The segmentation model is set up with U-Net++ architecture using ResNet101 as the feature extractor. The mask of each segmented patch will be merged to get a single binary mask and it will be quantified to get the final result. Pathologists can feed the histopathology images captured from the digital microscope into this tool and the output will be the segmented image and the total area of this segmented part. Different other methods have been proposed in this research for the segmentation task as this is the only AI-enabled part of this study. One among them is a segmen- tation model using Autoencoder where the training of the model is done with a single histopathology image. Another proposed model is designed using SegFormer, which is a transformer-based, encoder-decoder architecture. The SegFormer model also can be used to develop Necrosis-ML by replacing the U-Net architecture mentioned above. Another major contribution of this research is the dataset created in this study, named as NecrosisDB. This dataset contains 900 patches of images that include var- ious morphologies of necrosis, tumors, fibrosis, and other frequently occurring tissue elements. The major hurdle in any research in the medical domain is the unavailability of the annotated datasets, but in this study, we managed to create a fully supervised annotated dataset that has been annotated using experienced pathologists. This dataset has been made publicly accessible for research or study purpose based on some terms and conditions. It can be accessed from the link,