Image Quality Assessment Pipeline and Semi-Automated Annotation method for Synthetic Data

Abstract: Predicting human emotions through facial expression, particularly in relation to medication field such as clinical trial settings, has garnered scientific interest in recent years due to significant understanding of the impact of treatment on emotions and social functioning. This thesis aims to improve performance of a FER model using large scale of synthetic data. A FER classification neutral network’s performance is validated to accurately detect Action Units (AUs) in human facial images. To select the high-quality images among a pool of synthetic data, a Training Data Selection (TDS) pipeline is utilized, incorporating both no-reference and reference Image Quality Assessment (IQA) metrics. Furthermore, this thesis contributes through the development of a semi-automated annotation method, which offers an efficient approach to leverage an minimal amount of human annotation for labeling of a large number of images depicting various AUs. The proposed methodology incorporates seed tracking embedded in image names as a means to annotate the images. By integrating this annotation method with synthetic data generation, it minimizes the need for labor-intensive manual efforts and enables streamlined synthetic data annotation. Increasing the number of synthetic images to over 40,000, the model’s classification performance shows moderate improvement. Namely, the enhanced FER model performance outperforms or show the same result compared to the baseline result for the majority of the classes. This outcome highlights the efficacy of utilizing the TDS pipeline using IQA in conjunction with the semi-automated annotation method in improving the overall performance of the classification model. The model achieves a range of ROC AUCs that vary between 0.80 and 0.92 over six AUs for cross validation. These findings shed light on the challenges and limitations associated with using synthetic data for FER models. The findings also emphasize the need for further research to enhance the accuracy and reliability of synthetic data in this domain and the need for more accurate annotation of data with minimal interventions of human annotators.