Semantic Similarity Comparison of Political Statements by ChatGPT and Political Representatives

Abstract: ChatGPT is a recently released chatbot that through the use of deep learning can generate human-like statements on a variety of topics. Deep learning models have a potential to affect politics. They can for instance be used as a source for political information or be used to create and spread political messages. ChatGPT is itself able to describe the stances of different political parties and can generate political messages based on these stances. In this thesis, a semantic similarity program, utilizing the models Stanza and Sentence-BERT, is implemented. This program is used to compare the semantic similarity of political statements and information generated by ChatGPT to authentic statements and information written by Swedish political representatives prior to the 2022 general election. The results of the thesis demonstrate that ChatGPT with relatively high accuracy (over 60 % when three options are available) is able to correctly reflect the standpoints of Swedish political parties in specific political questions. When compared to authentic political information using semantic similarity, there is no discernible difference between the scores achieved by ChatGPT’s statements and the scores achieved by authentic statements from political representatives. This might reflect that ChatGPT performs well in semantically mimicking the style used by political representatives. Alternatively, the result could indicate limited usefulness of semantic similarity as a comparative method for political statements.