FACIAL FEATURE EXTRACTION USING LOCAL BINARY PATTERN AND LOCAL TERNARY PATTERN WITH GRADIENT BASED ILLUMINATION NORMALIZATION

– This paper presents a novel method of facial feature extraction along with illumination normalization. To suppress the effect of illumination, a gradient based illumination normalization technique is used in the pre-processing stage. Facial features are extracted using two local feature extractors Local Binary Pattern and Local Ternary Pattern. Local Binary Pattern is a very efficient method of feature extraction which is insensitive to monotonic grayscale variations in the image. A face image is split into small blocks and LBP histograms are computed for each sub block and then concatenated into a single feature vector. Local Ternary Patterns is noise resistant 3-state version of LBP. The facial feature extraction process is performed on the two very popular face databases Extended Yale B and AR database. The feature vectors obtained from LBP and LTP are highly discriminative and useful for further recognition tasks.


I. INTRODUCTION
Face recognition is a growing research area spanning various fields such as Image Processing, Cognitive science, Pattern Recognition, Neuroscience, Computer Vision etc. Human ability to recognize faces and complex patterns is incredible. It does not even get affected with the passage of time. Therefore we need to develop the systems as robust as humans in recognizing human faces. The face gives an ample of information about human behavior and is extremely hard to be falsified during recognition. A face recognition system is composed of three components viz. detection, feature extraction and face recognition [1]. In the detection phase, a pre-processed image is obtained from the input image by carrying out processes such as face edge detection, localization, segmentation etc. Feature extraction phase involves the extraction of image features such as statistical pixel features, visual features, algebraic features or transform coefficient features that convey the intrinsic properties of the image. The face recognition phase performs the classification of the extracted image features on the basis of certain criteria. Feature extraction is one of the most crucial steps in the face recognition system. Feature extraction process starts with an initial set of input data followed by the derivation of intended features which are more informative and nonredundant as compared to the whole lot of input data. The extracted features are then used in the subsequent learning and classification steps. Mostly, the input data is too large and redundant for processing in the further steps, so it can be modified into a reduced set of features which are more relevant and informative. Also to analyze a huge set of input data, large amount of memory and computational power is required which makes the algorithm cumbersome. The facial feature extraction techniques are broadly classified into following categories:

Geometric feature based methods
This is the most primitive method of facial feature extraction. Geometric feature based methods firstly detect the most significant facial features and then combines the distances among these features as well as other geometric characteristics into a single feature vector which can be then used in the further steps [2].

Template based methods
This is the most commonly used method of face recognition. Instead of using the most significant facial features as in geometric feature based methods, these methods use a feature vector that represents the entire face [2].

Correlation based methods
In the correlation based methods, normalized cross correlation coefficients are computed. Firstly the location of most prominent facial features such as nose, eyes, mouth is determined using set of templates and then the normalized correlation coefficients of these templates are computed [2].

Appearance based methods
The images are processed as two-dimensional patterns in these methods. The features considered in this approach are different from the simple facial features such as eyes, nose, and mouth. Any characteristic extracted from the face image is considered as a feature in this technique. It is the best performing technique for extraction of features from face images as it preserves the most significant information of the face image while rejects the unnecessary information [2].

II. REVIEW OF EXISTING TECHNIQUES
Feature extraction is an important and crucial issue in automatic face recognitions systems. Detection of the prominent facial features such as eyes, mouth and nose accurately is the most significant step in the face recognition algorithms. A lot of feature extraction techniques have been worked out by the researchers in the recent years. The most commonly used feature extraction techniques developed so far has been stated below: In [3] Timo Ahonen et al. proposed a new technique of face recognition in which both the shape and texture information is considered for the representation of face images. The face area is divided into small regions followed by the extraction of Local Binary Pattern (LBP) histograms and the histograms are concatenated into a single, spatially enhanced feature histogram which represents the face image efficiently. For the recognition purpose, the nearest neighbor classifier is used in the computed feature space with a dissimilarity measure as Chi Square. In [4] Terrence Chen et al. presented a new technique for recognizing faces under varying illumination conditions, including natural lighting condition, which is based on the logarithmic total variation (LTV) model. In the LTV model, a single face image is factorized and the illumination invariant facial structure is obtained, which is then used for the purpose of face recognition. The advantages of this model are (1) No lighting assumption is required. (2)No training process is required. In [5] S. Liao et al. came up with the idea of extraction of a set of features in a texture image by using dominant local binary patterns (DLBP) and the circularly symmetric Gabor filter responses. The most frequently occurring patterns are used by the dominant local binary pattern method so as to obtain the more descriptive textural information. The additional global textural information is supplied to DLBP features by the Gabor-based features. In [6] Amirhosein Nabatchian et al. proposed a method which is based on Illumination-reflection model. Illumination invariants are achieved by employing local matching for classification purpose. Maximum filter is the best to obtain the reflectance part of the image which is illumination invariant. Entropy and Mutual information is used as weight factors for the adaptively weighted classifiers. Support vector machine and K-Nearest neighbors are used as classifiers. In [7] X. Tan et al. presented a modified form of local binary pattern (LBP) which is called local ternary pattern (LTP). LBP encodes the neighboring pixels by taking the center pixel as the threshold value for comparison with other pixels. But the LBP technique is more sensitive to noise. However LTP encodes the pixels into three values using a threshold constant.LTP is noise resistant and more information is preserved in this technique due to the splitting of a ternary pattern into two binary patterns.

III. METHODOLOGY
The proposed method consists of extracting the facial features of the image using two feature extraction techniques along with the initial preprocessing. The input images are first preprocessed using illumination normalization technique based on local gradient to obtain the illumination insensitive representation of the image [8]. The feature extraction process is carried out on the preprocessed image using Local Binary Pattern and its noise resistance modified version Local Ternary Pattern. The required output features obtained from these two techniques can be used further in the classification or recognition of faces.

(A) Pre-processing using local gradient based illumination normalization
Illumination is the most prominent factor leading to the limiting the performance of face recognition systems. Illumination effect results in non uniform intensities or diminished structures of the face image caused by the difference in texture of the object surface and the shadows cast due to the different directions of light sources. Different methods of image normalization are used to minimize the influence of the illumination variations on the face recognition system. In the proposed work, a method based on gradient domain processing is used to obtain the illumination insensitive image representation. The gradient domain processing takes into consideration the dependency between the pixels and hence the image representation so obtained is more discriminative. The illumination effect is suppressed significantly if the ratio of the gradient amplitude to the original image intensity is taken [8].

(B) Feature Extraction using Local Binary Pattern
Local binary pattern [9] is a very influential technique that describes the texture of a face image. In the past few years, LBP has been increasingly used in various computer vision and image processing tasks. The features extracted using LBP are highly discriminative as a result of the different levels of locality involved. LBP is a very efficient technique due to its high speed of computation and is also robust against illumination variations.
In the Local Binary Pattern feature extraction technique, each pixel is characterized by the decimal numbers which are called Local Binary Patterns or LBP codes. These LBP codes encode the local structure around each pixel. The various steps involved in generating LBP features are stated as follows: 1. The LBP technique works in a 3x3block of image pixels.
2. The center pixel value is subtracted each of its eight neighbors in a 3x3 neighborhood and the same is repeated for each and every pixel of the image.
3. If the result of the comparison is negative, the corresponding pixel is assigned the value '0' and if it is positive the pixel is assigned a value '1'.
4. All the obtained binary codes are concatenated in a clockwise direction beginning from the left one at the top or bottom to form a binary number.
5. These binary numbers are then converted to decimal values that serve as labels for each pixel. These binary numbers are called LBP codes.

(C)Feature Extraction Using Local Ternary Pattern
Local Ternary Pattern [10] is a noise resistant version of Local Binary Pattern. Both the techniques are used for encoding the intensity difference between the center pixel and its neighbors. LBP is sensitive to noise because even a small gray level change of the center pixel may result in different codes for a neighborhood in an image, particularly for the smooth regions. LTP overcomes the noise sensitivity issue in LBP by encoding the small difference in pixels into a third state. Instead of thresholding the pixels into 0 and 1as in LBP, LTP uses a threshold constant to threshold the pixels into three values. Considering c as the value of center pixel, k as the threshold constant and p as the neighboring pixel, the result of thresholding is: where k is a given threshold constant and it makes the LTP code more robust against noise. In this way, one of the three values are assigned to each of the thresholded pixel by LTP. A ternary pattern is obtained by combining the neighboring pixels after thresholding. If the histogram of these ternary values is computed, it will result in a large range, so the ternary pattern is split into two binary patterns. A descriptor double the size of LBP is obtained by concatenating the histograms of these two binary patterns for each cell.

IV. CONCLUSION
In this work, facial feature extraction process is carried out using the illumination invariant descriptors Local Binary Pattern and its noise resistant modified version Local Ternary Pattern. The illumination effect is compensated by using local gradient based illumination normalization technique in the pre-processing stage. The facial features of two most popular face databases Extended Yale B and AR database are extracted using the proposed method. This method extracts the highly correlated facial image features while removing the irrelevant features. LBP is invariant to the monotonic grayscale variations while LTP is more noise resistant which results in an efficient feature extraction algorithm.