Multiple Classifier Systems: First International Workshop, MCS 2000 Cagliari, Italy, June 21-23, 2000 Proceedings

Many theoretical and experimental studies have shown that a multiple classi'er system is an e'ective technique for reducing prediction errors 9,10,11,20,19]. These studies identify mainly three elements that characterize a set of cl- si'ers: -Therepresentationoftheinput(whateachindividualclassi'erreceivesby wayofinput). -Thearchitectureoftheindividualclassi'ers(algorithmsandparametri- tion). - The way to cause these classi'ers to take a decision together. Itcanbeassumedthatacombinationmethodise'cientifeachindividualcl- si'ermakeserrors'inadi'erentway', sothatitcanbeexpectedthatmostofthe classi'ers can correct the mistakes that an individual one does 1,19]. The term 'weak classi'ers' refers to classi'ers whose capacity has been reduced in some way so as to increase their prediction diversity. Either their internal architecture issimple(e.g., theyusemono-layerperceptronsinsteadofmoresophisticated neural networks), or they are prevented from using all the information available. Sinceeachclassi'erseesdi'erentsectionsofthelearningset, theerrorcorre- tion among them is reduced. It has been shown that the majority vote is the beststrategyiftheerrorsamongtheclassi'ersarenotcorrelated.Moreover, in real applications, the majority vote also appears to be as e'cient as more sophisticated decision rules 2,13]. Onemethodofgeneratingadiversesetofclassi'ersistoupsetsomeaspect ofthetraininginputofwhichtheclassi'erisrather unstable. In the present paper, westudytwodistinctwaystocreatesuchweakenedclassi'ers;i.e.learning set resampling (using the 'Bagging' approach 5]), and random feature subset selection (using 'MFS', a Multiple Feature Subsets approach 3]). Other recent and similar techniques are not discussed here but are also based on modi'cations to the training and/or the feature set 7,8,12,21].