FasterCellpose: Knowledge Distillation for Efficient Fluorescence Nuclear Segmentation

Endocrine-disrupting chemicals (EDCs) are pollutants that interfere with hormone-receptor binding. In particular, agricultural pesticides have been linked to prostate cancer. To study their effects and the potential treatments, biologists use 3D in vitro organoids, captured by confocal microscopy with nuclear biomarker. Cell segmentation models are employed by biologists to produce a map that identifies the objects inside the organoid. The label masks simplify experts’ analysis. Although effective, the current models are computationally costly. Many laboratories can not afford high-performance GPUs. Moreover, labeled datasets are scarce and manual annotation is time-consuming. This thesis aims to achieve accurate cell 3D segmentation with a compact neural network on CPU-only hardware. For this purpose, a lightweight version of the well-known Cellpose is presented: FasterCellpose. The baseline architecture was simplified by using more efficient convolutions, removing unnecessary layers and halving the number of channels per depth scale. The resulting CNN model is 56x smaller and 8.4x faster than Cellpose for 3D segmentation on CPU. To preserve baseline performance, knowledge-distillation was employed to transfer pre-trained Cellpose “teacher” knowledge to the compact “student”. This work highlights the potential of model compression for both 2D and 3D cell segmentation. Furthermore, FasterCellpose can generalize like the baseline on external datasets. These results pave the way for resource-efficient, high-quality segmentation on 3D confocal images of organoids.