Proliferation Index estimation in Immunohistochemical images via Deep Learning: Stain Normalization Impact assessment in Nuclei Segmentation and Classification

In pathology, immunohistochemical (IHC) biomarkers are essential for characterizing cancers and improving therapy responses through targeted approaches. The proliferation index (PI) plays a vital role in histopathologic diagnostics, especially for tumors, as it reflects the rate of cell proliferation within a tissue sample. Therefore, PI is a crucial metric in pathology, and it is determined by measuring diaminobenzidine (DAB) intensity expression through immunohistochemistry. Typically, PI is visually assessed by pathologists through the examination of tissue samples. However, this manual counting method is suboptimal due to its significant intra- and inter-observer variability and the time-consuming nature of the process. Accurately distinguishing between proliferating (immuno-positive) and non-proliferating (immuno-negative) cells in breast tissue is essential for precise diagnosis and effective treatment planning. This thesis project addresses these issues through the implementation of a deep learning-based method for the segmentation and classification of nuclei within IHC-stained images. Specifically, the research explores the application of the Centripetal Direction Network (CDNet), a novel deep learning model, for nuclear instance segmentation. In the first stage of the study, the CDNet model was utilized to accurately segment nuclei in IHC-stained images. This model leverages the unique centripetal direction approach to improve segmentation accuracy, enabling precise identification and isolation of nuclear instances. Following the instance segmentation, the second stage involved classifying the previously segmented nuclei based on their staining characteristics. The classification process analyzed the color spaces of immunopositive nuclei (stained with DAB) and immunonegative nuclei (stained with hematoxylin counterstain). Subsequently, the proliferation index for each image was automatically computed. Additionally, the study aimed to investigate the impact of stain normalization on the performance of the CDNet model in segmentation tasks, as deep learning network performance in digital pathology is often hampered by variations in stain and scanner settings. The results indicated that stain normalization improved the accuracy of instance segmentation, demonstrating the importance of preprocessing steps in enhancing the performance of deep learning models. The final aspect of this research involved validating the proposed method by comparing the results with the ground truth provided by a pathologist. Each aspect of the framework was carefully validated: segmentation performance was evaluated using quantitative metrics, and the PI assessment was validated by examining its correlation with the provided ground truth values. Additionally, the results were compared with the PI values provided by the pathologist. In conclusion, by presenting both the original and normalized images to the pathologist, the evaluation demonstrated a statistically significant improvement in image quality and faster times to diagnosis for normalized images compared to original ones.