Developing and evaluating a style guide for chatbots deployed in a technical setting

Abstract: This study evaluates the perceived credibility of a technical chatbot based on its communication style - the way it interacts with its users embodied through text and emojis. A chatbot’s initial communication style was compared to a humble version. The humble communication style was developed from a design workshop held together with six participants and is presented in this paper as a design style guide. The perceived credibility was divided into six dimensions; Competence, Goodwill, Honesty, Predictability, Reputation, and Trustworthiness. The results from the evaluation of the two chatbot versions showed that the credibility was, in general, perceived higher for the chatbot using a humble communication style. Two exceptions were found; (1) the dimension of Trustworthiness stayed at the same level between the versions, and (2) the dimension of Goodwill was perceived higher for the chatbot not using the humble communication style. The satisfaction with the chatbot was measured and resulted in an NPS of 17 for the chatbot using the humble communication style compared to a negative score of -16 for the chatbot not using it. This study found that a more humble communication style would not harm the perceived credibility of a technical professional chatbot.