Human-in-the-loop Computing : Design Principles for Machine Learning Algorithms of Hybrid Intelligence

Abstract: Artificial intelligence (AI) is revolutionizing contemporary industries and being applied in application domains ranging from recommendation systems to self-driving cars. In scenarios in which humans are interacting with an AI, inaccurate algorithms could lead to human mistreatment or even harmful events. Human-in-the-loop computing is a machine learning approach desiring hybrid intelligence, the combination of human and machine intelligence, to achieve accurate and interpretable results. This thesis applies human-in-the-loop computing in a Design Science Research project with a Swedish manufacturing company to make operational processes more efficient. The thesis aims to investigate emerging design principles useful for designing machine learning algorithms of hybrid intelligence. Hereby, the thesis has two key contributions: First, a theoretical framework is built that comprises general design knowledge originating from Information Systems (IS) research. Second, the analysis of empirical findings leads to the review of general IS design principles and to the formulation of useful design principles for human-in-the-loop computing. Whereas the principle of AI-readiness improves the likelihood of strategical AI success, the principle of hybrid intelligence shows how useful it can be to trigger a demand for human-in-the-loop computing in involved stakeholders. The principle of use case-marketing might help designers to promote the customer benefits of applying human-in-the-loop computing in a research setting. By utilizing the principle of power relationship and the principle of human-AI trust, designers can demonstrate the humans’ power over AI and build a trusting human-machine relationship. Future research is encouraged to extend and specify the formulated design principles and employ human-in-the-loop computing in different research settings. With regard to technological advancements in brain-machine interfaces, human-in-the-loop computing might even become much more critical in the future.