NEURAL NETWORK BASED DEEP LEARNING AND ENSEMBLE TECHNIQUES FOR DATA CLASSIFICATION

In this paper, four neural network architectures are proposed for data classification. The four neural networks are constructed based on deep learning and ensemble architectures. Supervised and unsupervised learning paradigms are adopted. Stack of Supervised Neural Network (SSNN), Stack of Unsupervised Neural Network (SUNN), Ensemble of Supervised Neural Network (ESNN), and Ensemble of Unsupervised Neural Network (ESNN) are the proposed neural network classifiers. Australian credit approval data set from the University of California, Irvin is used to evaluate the classifiers. Supervised neural networks have produced more accurate classification results than unsupervised networks. Stack architectures are comparatively better than ensemble architectures. It is found from this research that the combination of supervised learning method and stack architecture leads to better performance.


I. INTRODUCTION
In the recent decades soft computing techniques have grabbed more attention of the researchers. Artificial neural networks, fuzzy logic and genetic algorithms are notable soft computing techniques and they are applied to solve various kinds of problem irrespective of domains. They are biologically inspired machine learning techniques. Supervised learning and unsupervised learning are the two significant paradigms of this soft computing approach. Deep learning neural networks and ensemble of neural networks are the advanced versions of regular methods. Deep learning can be achieved through the stack architecture and a high amount of training is required [1]. Deep neural networks are successfully applied to classification problems. Hierarchical fuzzy deep neural networks are applied for image categorization, financial data prediction and MRI segmentation of brain [2]. To improve the efficiency of deep neural networks, weight matrix parameterizing and low rank factorization are applied [3]. Diagnosis and classification of faults in semiconductor manufacturing process can be carried out by neural networks [4]. Deep convolutional features of neural network classifiers are used for classifying remote sensing images [5]. The effectiveness of deep learning in classification process is proved on remote sensing image data of lands and crops [6]. Extreme learning machines are built by stack architecture for solving various type of classification problems [7]. The complexity of deep learning neural network classifiers are discussed in the literature [8].
Application of the neural network based ensemble methods for classification problems are found better in terms of decreasing the error rate [9]. Ensembles of neural networks are applied for classification of seismic signals [10] and medical image classification [11]. Linear combination of multiple neural networks has improved the performance of classifier [12]. A combination of multiple neural networks for online pattern classification is attempted [13]. Hybrid methods such as Neuro-Fuzzy ensembles are producing better results in classification [14] [15]. Multilayer perceptron based ensembles can be used as classifiers [16] [17]. Clustering based classification is a novel approach to solve classification problems in some select domain [18]. When analyzing the literature, ensemble methods seemed better classifiers and they are applied on data related medical diagnosis also [19].

II. PROPOSED METHODS AND EXPERIMENTS
Australian credit approval data set is a bench mark data set provided by the UCI repository. It contains 690 records, 14 attributes and two classes. It is used here to evaluate the performance of proposed neural network architectures. The overall structure of the proposed approach is represented in Figure.  : Input weights vectors are fed into the first layer and in the second layer, the Euclidian distance among the weights is calculated to get better grouping. Maxnet activation function is applied repeatedly in the third, the fourth and the fifth layers in order to produce the accurate classification. The final classification result is derived through the output layer. : Input weights vectors are fed into the first layer and in the second layer, the Euclidian distance among the weights is calculated to get better grouping. Maxnet activation function is applied in the third layer. The final classification result is derived through the output layer. The consolidate result is produced by the ensemble at the end.

III. EXPERIMENTAL RESULTS
The classification accuracy and mean squared error (MSE) rate are calculated as follows.

Mean Squared Error of Classifiers
Australian Credit Approval Data Set

IV. CONCLUSION AND FUTURE SCOPE
The obtained classification accuracy of the proposed SSNN, SUNN, ESNN and EUNN are 86.3%, 84.6%, 85.9% and 85.2% respectively. The error rate of SSNN, SUNN, ESNN and EUNN are 0.1365, 0.1533, 0.1408, and 0.1472 respectively. Results indicate that among the proposed techniques, supervised methods are more efficient in classification when compared with unsupervised neural network. In future researches, hybrid soft computing techniques such as Neuro-fuzzy, Neuro-genetic, fuzzy-genetic may be applied to improve the performance of classifiers.