Measures of collaboration in CSCW: Usability and collective measures in remote and co-located problem-solving

Abstract: Since computers first came to be used in the society the role of the user has changed as hasthe research connected to it. Computer Supported Collaborative Work (CSCW) has focusedon several users and to evaluate existing collaborative systems. However, no full method isyet proposed. Authors state that there is a lack in usability and social studies. The aim of thisthesis has first been to find new measures for collaboration and second to test the validity ofthese measures on a set of experimental data. A literature review showed that good candidatesfor new measures were usability an collective measures. A careful observation of theexperimental data set helped to find seven candidate measures: Speaking, Monitoring,Looking at task description, Sketching, Erasing, Laughing and Task process that were appliedon an existing body of experimental data. The collection of experimental data stemmed fromvideo taped observations on a within-group design experiment involving eighteenparticipants. Nine pairs of subjects solved dot-to-dot puzzle problems under three conditions(first independent variable), two of these conditions were remote (SkypeVideo andCollaBoard); one condition was co-located (Co-located). There were three tasks with varyingdifficulty (second independent variable). Monitoring and Speaking were examined instatistical tests and the five other measures in descriptive analysis. Statistical significantdifference in means were found for tasks in Monitoring and Speaking. In Monitoring thedifference was due to the difference in task time. No significance was seen for conditionswhich could be due to a high variation as seen in the box-plots or that the conditions were toosimilar when solving the specific dot-to-dot puzzle problem. The descriptive data showedsimilar results. The Task process measure showed that in the Co-located condition most focuswas put on task-solving, in Skype Video the most focus was put on the whiteboard. In theCollaBoard condition Looking at task description had the most focus and the mostconversation and Laughing was seen. Most Laughing was also seen in the simple and hardtask. Also the most focus on the whiteboard was seen with the simple task. Most task-solvingwas done in the average task which implies that in an average task more focus can be put onsolving the task than on other things. The measures found in this study should be investigatedfurther as well as measures of CSCW usability and collectivity in general. A more usableCSCW system may save time, money, and our environment.