MAPREDUCE: INSIGHT ANALYSIS OF BIG DATA VIA PARALLEL DATA PROCESSING USING JAVA PROGRAMMING, HIVE AND APACHE PIG
Main Article Content
Abstract
Digital data which come from different sources like office, school, hospital, social media or machine generated data. Apache Hadoop is a software framework to store and process this enormous amount of data. Hadoop is using HDFS and MapReduce to store and process this huge volume of data. MapReduce is a programming model initiated by Google which can be written in different programming languages like Java, Python and Ruby. The main objective of this paper is to describe the concepts of MapReduce and showing the operation by using Java Program, Apache Pig and Hive. Hive and Apache Pig working on top layer of Hadoop ecosystem and provide the level of abstraction to run the MapReduce jobs. We write MapReduce program in Java to find anagram words from input files, group them together and save the result in output file. At the end we perform the operation in HiveQL (Hive query language) and Pig Latin Script and showing the backend process in MapReduce job.
Downloads
Download data is not yet available.
Article Details
Section
Articles
COPYRIGHT
Submission of a manuscript implies: that the work described has not been published before, that it is not under consideration for publication elsewhere; that if and when the manuscript is accepted for publication, the authors agree to automatic transfer of the copyright to the publisher.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work
- The journal allows the author(s) to retain publishing rights without restrictions.
- The journal allows the author(s) to hold the copyright without restrictions.