Virtual H&E Staining Using PLS Microscopy and Neural Networks

Abstract: Histopathological examination, crucial in diagnosing diseases such as cancer, traditionally relies on time- and resource-consuming, poorly standardized chemical staining for tissue visualization. This thesis presents a novel digital alternative using generative neural networks and a point light source (PLS) microscope to transform unstained skin tissue images into their stained counterparts. This approach utilizes PLS-microscopy’s unique illumination angles, providing more structural information about a sample and thereby enhancing a neural network’s ability to produce accurate, virtually stained images. Two matched datasets, each containing paired unstained and chemically stained tissue images, were used for supervised training of several networks. One dataset comprised healthy tissue, while the other, in addition to healthy tissue, included basal and squa- mous cell carcinomas. Given the limited scope of this master’s thesis, which constrained data acquisition, these datasets were relatively small, potentially impacting the general- izability of the model. The project explored the virtual staining capabilities of UNet and DenseUNet architectures, focusing on network depth and input channels. Variations in activation functions, upsampling blocks, and attention gates were tested, alongside the development of Relativistic Generative Adversarial Network (RGAN) models. Quantitative evaluation using standard metrics and qualitative assessment by pathologists and other medical professionals demonstrated the potential of PLS microscopy in virtual staining. The final model, based on RGAN, achieved superior staining accuracy with a structural similarity (SSIM) score of 0.799, significantly outperforming traditional bright field imaging (SSIM 0.631). However, the limited diversity and size of the datasets may have inflated these scores and highlight the need for caution in interpreting the results. The pathologists and medical professionals found virtually stained images indistinguishable from their chemically stained counterparts, with average stain quality ratings of 6.40 out of 10 for virtual images, which did not differ significantly from the rating of 6.41 for chemically stained ones. The pathologists and medical professionals were also able to classify 95.83% of all images as healthy or containing cancerous tissue correctly. In conclusion, virtual staining using PLS-microscopy holds considerable promise, offering a more standardized and sustainable approach compared to chemical staining. This method has the potential to speed up diagnosis and facilitate further analysis using image analysis algorithms. Future research could expand this technique beyond skin tissues, enhancing its applicability across a broader range of histopathological examinations.