EFFECT ON NAIVE BAYES CLASSIFIER OF FEATURE TRANSFORMATION: AN EMPIRICAL STUDY ON DIVERSE DATASETS

Abstract

Naïve Bayes classifiers are widely used for classification tasks because of their simplicity and for its efficiency along with strong theoretical foundation. However, their operational limitations stem from both the independence assumption concerning features and data distribution characteristics. The research investigates multiple techniques which apply feature transformations to improve both accuracy and stability of Gaussian Naïve Bayes classifiers. A comprehensive assessment of log transformation together with polynomial feature expansion on benchmark datasets consisting of Iris, Wine, Diabetes and Breast Cancer datasets occurred in this research. An evaluation of these transformations used cross-validation accuracy as well as precision, recall and F1-Score to assess their effects. Experimental outcomes show that distributions with skewed data can become more suitable for classification after log transformation while polynomial feature expansion improves feature characteristics to produce enhanced decision boundaries. In the Breast Cancer dataset Gaussian Naive Bayes with log-transformation outperformed the baseline model by producing superior recall and F1-Score results that are essential for medical diagnostics. The Wine dataset showed improved accuracy outcomes when using polynomial feature expansion because this method effectively extracts information about feature interactions which enhance class separability. Relevant preprocessing methods applied to Naïve Bayes classification yield improved predictive performance according to the results of this study. Future investigations will expand these transformation methods across different probabilistic classifiers as well as study their results within high-dimensional and imbalanced datasets.