From Software Engineer to being a Machine Learning Engineer - A Study on Skills and Responsibilities

Abstract: Background: With significant advancements in Machine Learning, the most advancing technology in multiple domains regardless of the field. It has primarily been the critical component of artificial intelligence and is used in a wide range of artificial Intelligence applications. In the industry, there is growing importance on machine learning, as organisations are actively looking for Machine Learning Experts who develop and create self-running AI systems that will automate processes; With the explosion of data in recent years and the increase in computing power, machine-learning systems have become much more capable. Thus the demand for machine learning skills is high in the job market due to the need for more advanced technology.   Objectives: Our thesis aims to provide a comprehensive understanding of primary machine-learning skills that are required by organizations to engineer a machine-learning system. The main objective of this research is to identify the important skills required to engineer a ML system; the important responsibilities looked for in a Machine Learning engineer. Methods: There are two parts of the methodology involved in this research. The first part of the research employs the method of Archival Research. In this method, we extracted the 3.4 million job ads dataset. We have got 6497 job ads related to machine learning. The Spacy Natural Language processing automated processing tool will be used to process the job advertisements and identify the required skills and responsibilities. After the skills and responsibilities are obtained, a qualitative survey is conducted on industrial expertise in machine learning to identify the most important skills that are needed and the most looked-for responsibilities for a ML engineer, which is the second part. Conclusions: The findings of this research have the potential to open up numerous opportunities for individuals who aspire to learn about machine learning systems. This includes software engineers and anyone else who wishes to gain a comprehensive understanding of the skills and responsibilities required to work on machine learning systems. The insights gained from this study may serve as a roadmap to transition into the machine-learning field and can also contribute to advancements in the field.