Does Changing the Wording or Scale on AttrakDiff Impact the User Assessment?

Abstract: This paper applies two main changes to a usability questionnaire called AttrakDiff, to challenge its validity in measuring user satisfaction levels. Modifying a standardized questionnaire is an uncommon practice in HCI; however, it occurs every so often. Although AttrakDiff is a valuable tool to measure the user assemessment of the attractiveness of an interactive system, it is not as widely used as the SUS or the CSUQ. There is a research gap in challenging AttrakDiff’s validity and quality, which is the main inspiration for this paper. The aim is to investigate whether changing the scale and wording impact the user assessment of interactive systems. An experiment with four groups of people, with ten people in each group, was carried out in order to challenge the standardized version of AttrakDiff. Each group was assigned a different questionnaire version, and their assessments were measured accordingly. The four groups consist of G1 with the original version, G2 with changed words, G3 with altered scale, and lastly, G4 with both changed words and scale. A total of 196 comparisons were conducted in the statistical analysis tool Jamovi. The results show that changing neither the scale nor the wording did cause a significant difference in the user assessment. However, whether a word is positive- or negative-toned appears to make a difference, as that affects a user’s perception of how a word is defined.