EFFICIENT PROCESSING OF JOB BY ENHANCING HADOOP MAPREDUCE FRAMEWORK

Article Sidebar

PDF
Published: Aug 28, 2017
DOI: https://doi.org/10.26483/ijarcs.v8i7.4256
Keywords:
Big Data, Data flow, Energy Efficiency, Map Reduce, Performance, Resource Allocation, Scheduling

Main Article Content

Ayesha Saad Khan
Integral University
M. Akheela Khanum

Abstract

Cloud Computing uses Hadoop framework for processing BigData in parallel. The Hadoop Map Reduce programming paradigm used in the context of Big Data, is one of the popular approaches that abstract the characterstics of parallel and distributed computing which comes off as a solution to Big Data. Improving performance of Map Reduce is a major concern as it affects the energy efficiency. Improving the energy efficiency of Map Reduce will have significant impact on energy savings for data centers. There are many parameters that influence the performance of Map Reduce . Various parameters like scheduling, resource allocation and data flow have a significant impact on Map Reduce performance. Cloud Computing leverages Hadoop framework for processing BigData in parallel. Hadoop has certain limitations that could be exploited to execute the job efficiently. Efficient resource allocation remains a challenge in Cloud Computing MapReduce platforms. We propose a methodology which is an enhanced Hadoop architecture that reduces the computation cost associated with BigData analysis.

Downloads

Download data is not yet available.

Article Details

Issue
Vol. 8 No. 7 (2017): July-August 2017
Section
Articles

COPYRIGHT

Submission of a manuscript implies: that the work described has not been published before, that it is not under consideration for publication elsewhere; that if and when the manuscript is accepted for publication, the authors agree to automatic transfer of the copyright to the publisher.

Authors who publish with this journal agree to the following terms:

  • Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
  • Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
  • Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work
  • The journal allows the author(s) to retain publishing rights without restrictions.
  • The journal allows the author(s) to hold the copyright without restrictions.

References

Eugen Feller,Lavanya Ramakrishnan,Christine Morin,†Performance and energy efficiency of big data applications in cloud environments: A Hadoop case studyâ€, Journal of Parallel and Distributed Computing,Elsevier (2015)

Mukhtaj Khan, Yong Jin, Maozhen Li, Yang Xiang and Changjun Jiang, “Hadoop Performance Modeling for Job Estimation and Resource Provisioningâ€, IEEE Transactions on Parallel and Distributed Systems.

Javier Conejero, Omer Rana, Peter Burnap, Jeffrey Morgan, Blanca Caminero, Carmen Carrión,†Analyzing Hadoop power consumption and impact on application QoSâ€, Future Generation Computer Systems 55 (2016)

Jacob Leverich, Christos Kozyrakis†On the energy (in)efficiency of Hadoop clustersâ€, Volume 44 Issue 1,January2010, Pages61-65 ,ACM New York, NY, USA

Rini T. Kaushik, Milind Bhandarkar†GreenHDFS: Towards An Energy-Conserving, Storage-Efficient, Hybrid Hadoop Compute Clusterâ€, HotPower'10 Proceedings of the 2010 international conference on Power aware computing and systems, Article No. 1-9, Vancouver, BC, Canada

Yanpei Chen, Archana Ganapathi†GreenHDFS: Towards An Energy-Conserving, Storage-Efficient, Hybrid Hadoop Compute Clusterâ€, HotPower'10 Proceedings of the 2010 international conference on Power aware computing and systems, Article No. 1-9, Vancouver, BC, Canada

Zhuo Tang, Lingang Jiang, Junging Zhou, Kenli Li, Keqin Li “A self-adaptive scheduling algorithm for reduce start timeâ€, Future Generation Computer System, Volumes 43–44, Pages 51–60 (2015)

Weikuan Yu, Yandong Wang, Xinyu Que, Cong Xu “Virtual Shuffling for Efficient Data Movement in MapReduceâ€, IEEE Transactions on Computers, Volume: 64, Issue: 2 (2015)

Kumar KA, Konishetty VK, Voruganti K, Rao GVP. CASH: Context Aware Scheduler for Hadoop. In: Proceedings of the International Conference on Advances in Computing, Communications and Informatics.

CloudSuite 1.0, Web page at http://parsa.epfl.ch/cloudsuite/cloudsuite.html (Last access: 26.06.14).

Tian C, Zhou H, He Y, Zha L. A dynamic MapReduce scheduler for heterogeneous workloads. In: 8th International Conference on Grid and Cooperative Computing. 2009. p. 218–24.