Quality in Crowdsourcing - How software quality is ensured in software crowdsourcing

Abstract: Crowdsourcing is a relatively new technique which aims to make a specific group of people contribute solutions to simple tasks or problems that are published online by some organization. For this they get some reward, which is usually economic in nature. This technique can be embraced by any kind of company, and since it is done online, it can turn out to be a bit problematic, especially when it comes to software development, because the whole process is out of the developing company’s hands. Some quality problems may arise during the process, such as a great amount of non-serious submissions and people presenting vague solutions because they are just trying to get the monetary reward. In order to make crowdsourcing successful these problems need to be solved, and companies which use this method for software development need to have some quality assurance for their products. This study tries to find out how companies using crowdsourcing deal with these problems and how they try to ensure some levels of quality in the final product. What we found is that companies embracing crowdsourcing use several methods in order to ensure a certain level of quality, such as rating, spam filters and reviews. There are many similarities in the underlying functions behind the methods each company uses such as motivating participants or finding the best solutions. These methods are applied at different stages throughout the crowdsourcing process. The exact relationships between the current use of these methods and the effect on software quality are not entirely apparent.