ADeFS: A Deep Forest Regression-Based Model to Enhance the Performance Based on LASSO and Elastic Net
In tree-based algorithms like random forest and deep forest, due to the presence of numerous inefficient trees and forests in the model, the computational load increases and the efficiency decreases. To address this issue, in the present paper, a model called Automatic Deep Forest Shrinkage (ADeFS)...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-12-01
|
| Series: | Mathematics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7390/13/1/118 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841549149773758464 |
|---|---|
| author | Zari Farhadi Mohammad-Reza Feizi-Derakhshi Israa Khalaf Salman Al-Tameemi Wonjoon Kim |
| author_facet | Zari Farhadi Mohammad-Reza Feizi-Derakhshi Israa Khalaf Salman Al-Tameemi Wonjoon Kim |
| author_sort | Zari Farhadi |
| collection | DOAJ |
| description | In tree-based algorithms like random forest and deep forest, due to the presence of numerous inefficient trees and forests in the model, the computational load increases and the efficiency decreases. To address this issue, in the present paper, a model called Automatic Deep Forest Shrinkage (ADeFS) is proposed based on shrinkage techniques. The purpose of this model is to reduce the number of trees, enhance the efficiency of the gcforest, and reduce computational load. The proposed model comprises four steps. The first step is multi-grained scanning, which carries out a sliding window strategy to scan the input data and extract the relations between features. The second step is cascade forest, which is structured layer-by-layer with a number of forests consisting of random forest (RF) and completely random forest (CRF) within each layer. In the third step, which is the innovation of this paper, shrinkage techniques such as LASSO and elastic net (EN) are employed to decrease the number of trees in the last layer of the previous step, thereby decreasing the computational load, and improving the gcforest performance. Among several shrinkage techniques, elastic net (EN) provides better performance. Finally, in the last step, the simple average ensemble method is employed to combine the remaining trees. The proposed model is evaluated by Monte Carlo simulation and three real datasets. Findings demonstrate the superior performance of the proposed ADeFS-EN model over both gcforest and RF, as well as the combination of RF with shrinkage techniques. |
| format | Article |
| id | doaj-art-753d4c9234f647cdad0de646d6702e85 |
| institution | Kabale University |
| issn | 2227-7390 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Mathematics |
| spelling | doaj-art-753d4c9234f647cdad0de646d6702e852025-01-10T13:18:18ZengMDPI AGMathematics2227-73902024-12-0113111810.3390/math13010118ADeFS: A Deep Forest Regression-Based Model to Enhance the Performance Based on LASSO and Elastic NetZari Farhadi0Mohammad-Reza Feizi-Derakhshi1Israa Khalaf Salman Al-Tameemi2Wonjoon Kim3Computerized Intelligence Systems Laboratory, Department of Computer Engineering, University of Tabriz, Tabriz 51666, IranComputerized Intelligence Systems Laboratory, Department of Computer Engineering, University of Tabriz, Tabriz 51666, IranComputerized Intelligence Systems Laboratory, Department of Computer Engineering, University of Tabriz, Tabriz 51666, IranDivision of Future Convergence (HCI Science Major), Dongduk Women’s University, Seoul 02748, Republic of KoreaIn tree-based algorithms like random forest and deep forest, due to the presence of numerous inefficient trees and forests in the model, the computational load increases and the efficiency decreases. To address this issue, in the present paper, a model called Automatic Deep Forest Shrinkage (ADeFS) is proposed based on shrinkage techniques. The purpose of this model is to reduce the number of trees, enhance the efficiency of the gcforest, and reduce computational load. The proposed model comprises four steps. The first step is multi-grained scanning, which carries out a sliding window strategy to scan the input data and extract the relations between features. The second step is cascade forest, which is structured layer-by-layer with a number of forests consisting of random forest (RF) and completely random forest (CRF) within each layer. In the third step, which is the innovation of this paper, shrinkage techniques such as LASSO and elastic net (EN) are employed to decrease the number of trees in the last layer of the previous step, thereby decreasing the computational load, and improving the gcforest performance. Among several shrinkage techniques, elastic net (EN) provides better performance. Finally, in the last step, the simple average ensemble method is employed to combine the remaining trees. The proposed model is evaluated by Monte Carlo simulation and three real datasets. Findings demonstrate the superior performance of the proposed ADeFS-EN model over both gcforest and RF, as well as the combination of RF with shrinkage techniques.https://www.mdpi.com/2227-7390/13/1/118deep forestdeep learningcascade forestmachine learningelastic netLASSO |
| spellingShingle | Zari Farhadi Mohammad-Reza Feizi-Derakhshi Israa Khalaf Salman Al-Tameemi Wonjoon Kim ADeFS: A Deep Forest Regression-Based Model to Enhance the Performance Based on LASSO and Elastic Net Mathematics deep forest deep learning cascade forest machine learning elastic net LASSO |
| title | ADeFS: A Deep Forest Regression-Based Model to Enhance the Performance Based on LASSO and Elastic Net |
| title_full | ADeFS: A Deep Forest Regression-Based Model to Enhance the Performance Based on LASSO and Elastic Net |
| title_fullStr | ADeFS: A Deep Forest Regression-Based Model to Enhance the Performance Based on LASSO and Elastic Net |
| title_full_unstemmed | ADeFS: A Deep Forest Regression-Based Model to Enhance the Performance Based on LASSO and Elastic Net |
| title_short | ADeFS: A Deep Forest Regression-Based Model to Enhance the Performance Based on LASSO and Elastic Net |
| title_sort | adefs a deep forest regression based model to enhance the performance based on lasso and elastic net |
| topic | deep forest deep learning cascade forest machine learning elastic net LASSO |
| url | https://www.mdpi.com/2227-7390/13/1/118 |
| work_keys_str_mv | AT zarifarhadi adefsadeepforestregressionbasedmodeltoenhancetheperformancebasedonlassoandelasticnet AT mohammadrezafeiziderakhshi adefsadeepforestregressionbasedmodeltoenhancetheperformancebasedonlassoandelasticnet AT israakhalafsalmanaltameemi adefsadeepforestregressionbasedmodeltoenhancetheperformancebasedonlassoandelasticnet AT wonjoonkim adefsadeepforestregressionbasedmodeltoenhancetheperformancebasedonlassoandelasticnet |