Data Duplication and Near Data Duplication Methods: A Review

Abstract

Search engines are the major component on the web for retrieving the information. But List of retrieved documents contains a high degree of duplicated and near document result. So in this case, there is a need to improve the performance of search results. For elimination of data duplicate and near duplicate document detection, search engine use data filtering algorithm to save user’s effort and time. In this paper, a detailed comparison of existing methods is presented.

Keywords: duplicate document, near duplicate pages, detection approaches