Improving recall of In situ sequencing by self-learned features and classical image analysis techniques

Image-based sequencing method to decode mRNA fragments directly in fixed tissue samples allows to carry out the gene expression profile preserving morphological and spatial information of cells and tissues. This approach called in situ sequencing makes it possible to directly visualize, at sub-cellular resolution, where in a tissue sample a given gene is active, to quantify its expression, and to distinguish among many different cell types at the same time. Such information are fundamental to gain a better understanding about tissue and disease development (such as cancer) and cells interplay. Since each gene is composed by a specific sequence of bases, the search is addressed to targeted sequences which must be decoded over multiple staining and imaging cycles, and retrieved by processing multichannel fluorescent biological images of the analyzed samples. However, signal density, high signal to noise ratio, and microscopes' resolution limits make decoding challenging. The state-of-art approach for signal decoding has led to low signal recall in efforts to maintain high sensitivity. The main issues related to the state-of-art technique concern difficulties in distinguishing signals in more dense regions, the lack of a proper handling for local misalignments among signals belonging to different sequencing cycles and the inability of processing 3D datasets. In this thesis a new approach has been implemented in order to face the state-of-art issues and increase recall at maintained sensitivity. Here signal candidates are included in the first processing steps and provided with their true-signal probability by an opportunely trained classifier. Signal candidates and their probability predictions are then fed to a decoding approach searching for signal candidates across sequencing cycles. Finally, the decoded sequences are provided with a quality measure indicating their reliability based on the classifier probabilities. In order to find the best solution, either a support vector machine and convolutional neural network have been tested as classifier. A window-based search has been designed for the sequence decoding. The developed sequence decoding method looks for the optimal paths representing the decoded signal sequences by combining intensity, probability and spatial distance. Multiple quality metrics have been tested to find out which one allowed to obtain the highest signal recall. All the possible combinations of the new proposed pipeline have been evaluated in relation of the state-of-art. Using the support vector machine as classifier has led to a consistent decrease in signal recall (20%) compared to the state-of-art pipeline. On the other hand, using the convolutional neural network has led to an improvement (31%). The obtained results demonstrated an evident advantage in using a classifier based on self-learned features and the need of a sequence decoding approach less dependent on signal probability predictions.The new proposed approach solves all the state-of-art issues and has the potential of significantly improve further analysis of spatial statistics in in situ sequencing experiments.