International Journal of Emerging Trends & Technology in Computer Science
A Motivation for Recent Innovation & Research
ISSN 2278-6856
www.ijettcs.org

Call for Paper, Published Articles, Indexing Infromation AIMS: Adaptive Improved MapReduce Scheduling in Hadoop, Authors : D.Vasudha, K.Ishthaq Ahamed , International Journal of Emerging Trends & Technology in Computer Science (IJETTCS), www.ijettcs.org
Volume & Issue no: Volume 6, Issue 4, July - August 2017

Title:
AIMS: Adaptive Improved MapReduce Scheduling in Hadoop
Author Name:
D.Vasudha, K.Ishthaq Ahamed
Abstract:
Abstract Hadoop is a complete eco-system of open source projects that provide us the framework to deal with big data. Let’s start by brainstorming the possible challenges of dealing with big data (on traditional systems) and then look at the capability of Hadoop solution. This approach lowers the risk of catastrophic system failure and unexpected data loss, even if a significant number of nodes become inoperative. Consequently, Hadoop quickly emerged as a foundation for big data processing tasks, such as scientific analytics, business and sales planning, and processing enormous volumes of sensor data, including from internet of things sensors. As an important extension of Hadoop, SAMR and ESAMR MapReduce scheduling algorithms take heterogeneous environment into consideration. However, it falls short of solving the crucial problem – poor performance due to the large data sets in which it computes progress of tasks. Consequently, neither Hadoop nor ESAMR schedulers are desirable in heterogeneous environment. To this end, we propose AIMS: an Adaptive Improved Scheduling Algorithm, which calculates progressof tasks dynamically and splits the files into multiple chunks, based on user inputs and adapts to the continuously varying environment automatically. When a job is committed, AIMS splits the input file into multiple chunks so that the process will be completed easily, then assigns them to a series of nodes. Meanwhile, it reads historical information which stored on every node and updated after every execution. Then, AIMSadjusts time weight of each stage of map and reducetasks according to the historical information respectively. Thus, it gets the progress of each task accurately and finds which tasks need backup tasks. What’s more, it identifies slow nodes and classifies them into the sets of slow nodes dynamically. According to the information of these slow nodes, AIMS will not launch backup tasks on them, ensuring the backup tasks will not be slow tasks any more. It gets the final results of the fine-grained tasks when either slow tasks or backup tasks finish first. The proposed algorithm is evaluated by extensive experiments over various heterogeneous environment. Experimental results show that AIMS significantly decreases the time of execution up to 19% compared with Hadoop’s scheduler and up to 10% compared with ESAMR scheduler.. Keywords: Map Reduce, Scheduling algorithm, He terogeneous environment, Self-adaptive, aims
Cite this article:
D.Vasudha, K.Ishthaq Ahamed , " AIMS: Adaptive Improved MapReduce Scheduling in Hadoop " , International Journal of Emerging Trends & Technology in Computer Science (IJETTCS), Volume 6, Issue 4, July - August 2017 , pp. 249-256 , ISSN 2278-6856.
Full Text [PDF]                          Home