A Comparison Study on Rule Extraction from Neural Network Ensembles, Boosted Shallow Trees, and SVMs
One way to make the knowledge stored in an artificial neural network more intelligible is to extract symbolic rules. However, producing rules from Multilayer Perceptrons (MLPs) is an NP-hard problem. Many techniques have been introduced to generate rules from single neural networks, but very few wer...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2018-01-01
|
| Series: | Applied Computational Intelligence and Soft Computing |
| Online Access: | http://dx.doi.org/10.1155/2018/4084850 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832559785105424384 |
|---|---|
| author | Guido Bologna Yoichi Hayashi |
| author_facet | Guido Bologna Yoichi Hayashi |
| author_sort | Guido Bologna |
| collection | DOAJ |
| description | One way to make the knowledge stored in an artificial neural network more intelligible is to extract symbolic rules. However, producing rules from Multilayer Perceptrons (MLPs) is an NP-hard problem. Many techniques have been introduced to generate rules from single neural networks, but very few were proposed for ensembles. Moreover, experiments were rarely assessed by 10-fold cross-validation trials. In this work, based on the Discretized Interpretable Multilayer Perceptron (DIMLP), experiments were performed on 10 repetitions of stratified 10-fold cross-validation trials over 25 binary classification problems. The DIMLP architecture allowed us to produce rules from DIMLP ensembles, boosted shallow trees (BSTs), and Support Vector Machines (SVM). The complexity of rulesets was measured with the average number of generated rules and average number of antecedents per rule. From the 25 used classification problems, the most complex rulesets were generated from BSTs trained by “gentle boosting” and “real boosting.” Moreover, we clearly observed that the less complex the rules were, the better their fidelity was. In fact, rules generated from decision stumps trained by modest boosting were, for almost all the 25 datasets, the simplest with the highest fidelity. Finally, in terms of average predictive accuracy and average ruleset complexity, the comparison of some of our results to those reported in the literature proved to be competitive. |
| format | Article |
| id | doaj-art-ac1b23340c484821b2bbb04903677c20 |
| institution | Kabale University |
| issn | 1687-9724 1687-9732 |
| language | English |
| publishDate | 2018-01-01 |
| publisher | Wiley |
| record_format | Article |
| series | Applied Computational Intelligence and Soft Computing |
| spelling | doaj-art-ac1b23340c484821b2bbb04903677c202025-02-03T01:29:13ZengWileyApplied Computational Intelligence and Soft Computing1687-97241687-97322018-01-01201810.1155/2018/40848504084850A Comparison Study on Rule Extraction from Neural Network Ensembles, Boosted Shallow Trees, and SVMsGuido Bologna0Yoichi Hayashi1Department of Computer Science, University of Applied Sciences and Arts Western Switzerland, Rue de la Prairie 4, 1202 Geneva, SwitzerlandDepartment of Computer Science, Meiji University, Tama-ku, Kawasaki, Kanagawa 214-8571, JapanOne way to make the knowledge stored in an artificial neural network more intelligible is to extract symbolic rules. However, producing rules from Multilayer Perceptrons (MLPs) is an NP-hard problem. Many techniques have been introduced to generate rules from single neural networks, but very few were proposed for ensembles. Moreover, experiments were rarely assessed by 10-fold cross-validation trials. In this work, based on the Discretized Interpretable Multilayer Perceptron (DIMLP), experiments were performed on 10 repetitions of stratified 10-fold cross-validation trials over 25 binary classification problems. The DIMLP architecture allowed us to produce rules from DIMLP ensembles, boosted shallow trees (BSTs), and Support Vector Machines (SVM). The complexity of rulesets was measured with the average number of generated rules and average number of antecedents per rule. From the 25 used classification problems, the most complex rulesets were generated from BSTs trained by “gentle boosting” and “real boosting.” Moreover, we clearly observed that the less complex the rules were, the better their fidelity was. In fact, rules generated from decision stumps trained by modest boosting were, for almost all the 25 datasets, the simplest with the highest fidelity. Finally, in terms of average predictive accuracy and average ruleset complexity, the comparison of some of our results to those reported in the literature proved to be competitive.http://dx.doi.org/10.1155/2018/4084850 |
| spellingShingle | Guido Bologna Yoichi Hayashi A Comparison Study on Rule Extraction from Neural Network Ensembles, Boosted Shallow Trees, and SVMs Applied Computational Intelligence and Soft Computing |
| title | A Comparison Study on Rule Extraction from Neural Network Ensembles, Boosted Shallow Trees, and SVMs |
| title_full | A Comparison Study on Rule Extraction from Neural Network Ensembles, Boosted Shallow Trees, and SVMs |
| title_fullStr | A Comparison Study on Rule Extraction from Neural Network Ensembles, Boosted Shallow Trees, and SVMs |
| title_full_unstemmed | A Comparison Study on Rule Extraction from Neural Network Ensembles, Boosted Shallow Trees, and SVMs |
| title_short | A Comparison Study on Rule Extraction from Neural Network Ensembles, Boosted Shallow Trees, and SVMs |
| title_sort | comparison study on rule extraction from neural network ensembles boosted shallow trees and svms |
| url | http://dx.doi.org/10.1155/2018/4084850 |
| work_keys_str_mv | AT guidobologna acomparisonstudyonruleextractionfromneuralnetworkensemblesboostedshallowtreesandsvms AT yoichihayashi acomparisonstudyonruleextractionfromneuralnetworkensemblesboostedshallowtreesandsvms AT guidobologna comparisonstudyonruleextractionfromneuralnetworkensemblesboostedshallowtreesandsvms AT yoichihayashi comparisonstudyonruleextractionfromneuralnetworkensemblesboostedshallowtreesandsvms |