Comparing Non-Bayesian Uncertainty Evaluation Methods in Chromosome Classification by Using Deep Neural Networks

Abstract: Chromosome classification is one of the essential tasks in karyotyping to diagnose genetic abnormalities like some types of cancers and Down syndrome. Deep convolutional neural networks have been widely used in this task, and the accuracy of classification models is exceptionally critical to such sensitive medical diagnoses. However, it is not always possible to meet the expected accuracy rates for diagnosis. So, it is vital to tell how certain or uncertain a model is with its decision. In our work, we use two metrics, entropy and variance, as uncertainty measurements. Moreover, three additional metrics, fail rate, workload, and tolerance range, are used to measure uncertainty metrics’ quality. Four different non-Bayesian methods: deep ensembles, snapshot ensembles, Test Time Augmentation, and Test Time Dropout, are used in experiments. A negative correlation is observed between the accuracy and the uncertainty estimation; the higher the accuracy of the model, the lower the uncertainty. Densenet121 with deep ensembles as the uncertainty evaluation method and variance as the uncertainty metric gives the best outcomes. Densenet121 provides a wider tolerance range and better separation between uncertain and certain predictions.