BPMN flows as variation points for end user development : from a UX perspective

Abstract: Context. How end user development can be enabled and made attainable through the use of a web based graphical user interface, in systems that contains logic for handling BPMN flows as variation points. Investigated from a UX and usability perspective. Objectives. Designing prototypes of such a web-based interface, and then evaluating the usability with regards to the usability attributes effectiveness and learnability, with the goal finding relevant usability issues & solutions as well as investigating how the two usability attributes affected the participants’ subjective satisfaction. Methods. Two prototype versions were implemented, one based upon the other. A usability inspection (expert evaluation) was performed after the first prototype version (named alpha) was finished and the second version (named beta) was built based on that feedback. The usability of the prototypes were then evaluated in usability test sessions using the think aloud method together with the SUS (System Usability Scale) questionnaire.The recorded data from the usability test sessions was analysed. Usability issues & solutions were noted, filtered, tagged and grouped by design principle with the goal of looking for patterns. SUS scores were calculated from the questionnaires. Additional factor analysis was performed on the SUS data to get separate usability and learnability scores. Results. The results consisted of SUS scores for both prototypes together with learnability and usability scores. Common and noteworthy usability issues & solutions grouped by design principle. Detailed appendixes with scenarios exemplifying recorded video & voice data for four of the most relevant test participants. Conclusions. The two prototypes were compared. Improvements to effectiveness and learnability was found to have a positive impact on the participants’ subjective satisfaction in the described context. Additionally, a number of usability issues and solutions was identified that could be of value when developing similar software. In summary, the following findings were made related to effectiveness: Adding constraints to the number of options available to users helped increase effectiveness. The lack of keyboard shortcuts was a deal breaker for many users and had a negative impact on effectiveness.. Consistency in navigation was more important than expected. The lack of functionality for saving drafts in the browser, without downloading, was something that most users expected and was surprised not to find. Several users expected their drafts to be automatically saved. They were frustrated when their changes were lost without warning. The lack of an undo function was also a big issue for users, causing problems with recovery. Giving immediate feedback with a notification popup after users had deployed a flow worked well and was easier to implement than expected. The connection tool that ship with bpmn-js was hard to learn and use for several users. Additionally, some learnability related findings: Sandbox intended to boost learnability but caused problems with effectiveness. The BPMN notation was not familiar to the test participants. Some training or introduction would have been necessary in a real usage scenario. When important functionality was concealed in submenus it was harder for users to learn how to operate the editor, specifically a problem with the context pad. Properties panel. Issues with visibility of single items due to tabs being too cluttered. Affordance issues on input/output parameters tab. Consistency issue with the bpmn.io logo being placed in the where users expected a save button. Consistency. Negative impact on learnability due to lack of keyboard shortcuts. Users likely to learn faster if they could use the shortcuts they already know. Placement of the download button not consistent with other webapps. Unconventional to only have drag and drop for upload and no button. Issues with visibility and consistency related to maximizing/minimizing editor in beta prototype. Familiarity findings related to deploy button colour and label might have affected learnability.