Investigating the structural diversity within a committee of classifiers and their generalization performance
Date
2009-10-30T11:35:13Z
Authors
Masisi, Lesedi Melton
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
This study investigates the measures of diversity within ensembles of classifiers. The use of
neural networks is carried out in measuring ensemble diversity by the use of statistical and
ecological methods and to some extent information theory. A new way of looking at ensemble
diversity is proposed. This ensemble diversity is called ensemble structural diversity, for this
study is concerned with the diversity within the structure of the individual classifiers forming an
ensemble and not via the outcomes of the individual classifiers. Ensemble structural diversity
was also induced within the ensemble by varying the structural parameters (learning parameters)
of the artificial machines (classifiers). The importance or the use of these measures was judged
by comparing the measure of structural diversity and the ensemble generalization performance.
This was done so that comparisons can be made on the robustness of the idea of structural
diversity and its relationship with ensemble generalization performance. It was found that
diversity could be induced by having ensembles with different structural and implicit (e.g
learning) parameters and that this diversity does influence the predictive ability of the ensemble.
This was concurrent with literature even though within literature ensemble diversity was viewed
from the output as opposed to the structure of the individual classifiers. As the structural
diversity increased so did the generalization performance. However there was a point where
structural diversity decreased the generalization performance of the ensemble, where from that
point onwards as the structural diversity increased the generalization performance decreased.
This makes sense because too much of diversity within the ensemble might mean no consensus
is reached at all. The disadvantages of comparing structural diversity and the generalization
performance (accuracy) of the ensemble are that: an ensemble can be structurally diverse even
though all the classifiers within the ensemble approximate the same function which means in this
case structural diversity is meaningless in terms of improving the accuracy of the ensemble. The
use of ensemble structural diversity measures in developing efficient ensembles still remains to
be explored. This study, however, has also shown that diversity can be measured from the
structural parameters and moreover reducing the abstractness of diversity by being able to
quantify structural diversity making it possible to map a relationship between structural diversity
and accuracy. It was observed that structural diversity does improve the accuracy of the
ensemble, however, within a limited region of structural diversity.