Policy-Driven YARN Launcher

Abstract: In recent years, there has been a rising demand for IT solutions that are capable to handle vast amount of data. Hadoop became the de facto software framework for distributed storage and distributed processing of huge datasets with a high pace. YARN is the resource management layer for Hadoop ecosystem which decouples the programming model from the resource management mechanism. Although Hadoop and YARN create a powerful ecosystem which provides scalability and flexibility, launching applications with YARN currently requires intimate knowledge of YARN’s inner workings. This thesis focuses on designing and developing support for a human-friendly YARN application launching environmen twhere the system takes responsibility for allocating resources to applications. This novel idea will simplify the launching process of an application and it will give the opportunity to inexperienced users to run applications over Hadoop.