Comparison of Evaluation Metrics in Classification Applications with Imbalanced Datasets

Title	Comparison of Evaluation Metrics in Classification Applications with Imbalanced Datasets
Publication Type	Conference Paper
Year of Publication	2008
Authors	Fatourechi, M., R. K. Ward, S. G. Mason, J. Huggins, A. Schlogl, and G. E. Birch
Conference Name	Machine Learning and Applications, 2008. ICMLA '08. 7th International Conference on
Pagination	777 -782
Date Published	dec.
Keywords	classification application, classifier testing, evaluation metrics, imbalanced datasets, Kappa coefficient, model selection, pattern classification
Abstract	A new framework is proposed for comparing evaluation metrics in classification applications with imbalanced datasets (i.e., the probability of one class vastly exceeds others). For model selection as well as testing the performance of a classifier, this framework finds the most suitable evaluation metric amongst a number of metrics. We apply this framework to compare two metrics: overall accuracy and Kappa coefficient. Simulation results demonstrate that Kappa coefficient is more suitable.
URL	http://dx.doi.org/10.1109/ICMLA.2008.34
DOI	10.1109/ICMLA.2008.34

The University of British Columbia