TRAFFIC MANAGEMENT USING MAPREDUCE FOR BIG DATA ANALYTICS
Main Article Content
Abstract
The MapReduce programming model simplifies large-scale processing on trade goods cluster by exploiting parallel map tasks and cut back tasks. though several efforts are created to boost the performance of MapReduce jobs, they ignore the network traffic generated within the shuffle part, that plays a crucial role in performance improvement.Historically, a hash perform is employed to partition intermediate knowledge among cut back tasks, which, however, isn't traffic-efficient as a result of configuration and knowledge size related to every key don't seem to be taken into thought. During this paper, we have a tendency to study to scale back network traffic value for a MapReduce job by coming up with a unique intermediate knowledge partition theme. what is more, we have a tendency to conjointly contemplate the aggregation placement downside, wherever every aggregation will cut back united traffic from multiple map tasks. A decomposition-based distributed algorithmic rule is projected to subsume the large-scale improvement downside for large knowledge application and a web algorithmic rule is additionally designed to regulate knowledge partition and aggregation in a very dynamic manner. Finally, in depth simulation results demonstrate that our proposals will considerably cut back network traffic value below each offline and on-line cases.
Downloads
Article Details
COPYRIGHT
Submission of a manuscript implies: that the work described has not been published before, that it is not under consideration for publication elsewhere; that if and when the manuscript is accepted for publication, the authors agree to automatic transfer of the copyright to the publisher.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work
- The journal allows the author(s) to retain publishing rights without restrictions.
- The journal allows the author(s) to hold the copyright without restrictions.
References
. Ms. Varalakshmi M N 1, Mrs. Shashirekha H, On Traffic-Aware Partition and Aggregation in Map Reduce for Big Data Applications, International Digital Library For Education & Research Volume 1,Issue 3, pp.1-7 Mar 2017.
. Huan Ke, Student Member, IEEE, Peng Li, Member, IEEE, Song Guo, Senior Member, IEEE, and Minyi Guo, Senior Member, IEEEOn Traffic-Aware Partition and Aggregation in Map Reduce for Big Data Applications. IEEE Transaction on Parallel and Distributed System (Volume: 27,Issue:3, pp. 818-828, March 1 2016.
. Y. Dinesh Reddy and A. Pio Sajin, An Efficient Network Traffic-Aware Partition for Big Data Application and Aggregation Techniques using Map Reduce. 2016 Vol. 9, pp.1-9, IJATIR 2016.
J. Dean and S. Ghemawat, “Mapreduce: simplified processing on massive clusters,†Communications of the ACM, vol. 51, no. 1, pp. 107 – 113, 2008.
W. Wang, K. Zhu, L. Ying, J. Tan, and L. Zhang, “Map task planning in mapreduce with knowledge locality: outturn and heavy- traffic optimality,†in INFOCOM, 2013 Proceedings IEEE.IEEE, 2013, pp. 1609– 1617.
F. Chen, M. Kodialam, and T. Lakshman, “Joint planning of process and shuffle phases in mapreduce systems,†in INFOCOM,2012 Proceedings IEEE. IEEE, 2012, pp. 1143 – 1151.
Y. Wang, W. Wang, C. Ma, and D.Meng, “Zput: A speedy knowledge uploading approach for the hadoop distributed filing system,†in Cluster Computing (CLUSTER),2013 IEEE International Conference on. IEEE, 2013, pp. 1 – 5.
T. White, Hadoop: the definitive guide: the definitive guide. †O’ReillyMedia, Inc.â€, 2009.[6] S. Chen and S. W. Schlosser, “Map-reduce meets wider forms of applications,†Intelanalysis Pittsburgh, Tech. Rep. IRP-TR-08-05, 2008.[7] J. Rosen, N. Polyzotis, V. Borkar, Y. Bu, M. J. Carey, M. Iimer, T. Condie, and R.Ramakrishnan, “Iterative mapreduce for giant scale machine learning,†arXiv preprintarXiv:1303.3517, 2013.
S. Venkataraman, E. Bodzsar, I. Roy, A. AuYoung, and R. S. Schreiber, “Presto:distributed machine learning and graph process with distributed matrices,†in Proceedingsof the eighth ACM European Conference on pc Systems. ACM, 2013, pp. 197 – 210.