Michael Green and Mattias Ohlsson
Comparison of standard resampling methods for performance estimation of a rtificial neural network ensembles
In Proceedings of The third international conference on Computational Intelligence in Medicine and Healthcare, Plymouth, England, eds. E. Ifeachor (2007)
Abstract:
Estimation of the generalization performance for
classification within the medical applications domain is
always an important task. In this study we focus on artificial
neural network ensembles as the machine learning technique. We
present a numerical comparison between five common resampling
techniques: k-fold cross validation (CV), holdout, using three
cutoffs, and bootstrap using five different data sets. The
results show that CV together with holdout 0.25 and 0.50
are the best resampling strategies for estimating the true
performance of ANN ensembles. The bootstrap, using the .632+
rule, is too optimistic, while the holdout 0.75
underestimates the true performance.
LU TP 07-16