A Novel Scheme for Improving Accuracy of KNN Classification Algorithm Based on the New Weighting Technique and Stepwise Feature Selection
K nearest neighbor algorithm is one of the most frequently used techniques in data mining for its integrity and performance. Though the KNN algorithm is highly effective in many cases, it has some essential deficiencies, which affects the classification accuracy of the algorithm. First, the effectiv...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
University of Tehran
2020-12-01
|
| Series: | Journal of Information Technology Management |
| Subjects: | |
| Online Access: | https://jitm.ut.ac.ir/article_78401_aa4ee9377b284f7eaf141c5a92adbe96.pdf |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849724847256502272 |
|---|---|
| author | Saeid Sheikhi Mohammad Taghi Kheirabadi Amin Bazzazi |
| author_facet | Saeid Sheikhi Mohammad Taghi Kheirabadi Amin Bazzazi |
| author_sort | Saeid Sheikhi |
| collection | DOAJ |
| description | K nearest neighbor algorithm is one of the most frequently used techniques in data mining for its integrity and performance. Though the KNN algorithm is highly effective in many cases, it has some essential deficiencies, which affects the classification accuracy of the algorithm. First, the effectiveness of the algorithm is affected by redundant and irrelevant features. Furthermore, this algorithm does not consider the differences between samples, which led the algorithm to have inaccurate predictions. In this paper, we proposed a novel scheme for improving the accuracy of the KNN classification algorithm based on the new weighting technique and stepwise feature selection. First, we used a stepwise feature selection method to eliminate irrelevant features and select highly correlated features with the class category. Then a new weighting method was proposed to give authority value to each sample in train dataset based on neighbor categories and Euclidean distances. This weighting approach gives a higher preference to samples that have neighbors with close Euclidean distance while they are in the same category, which can effectively increase the classification accuracy of the algorithm. We evaluated the accuracy rate of the proposed method and analyzed it with the traditional KNN algorithm and some similar works with the use of five real-world UCI datasets. The experiment results determined that the proposed scheme (denoted by WAD-KNN) performed better than the traditional KNN algorithm and considered approaches with the improvement of approximately 10% accuracy. |
| format | Article |
| id | doaj-art-573b5feb6aeb425da2215a72aa4d5e8b |
| institution | DOAJ |
| issn | 2008-5893 2423-5059 |
| language | English |
| publishDate | 2020-12-01 |
| publisher | University of Tehran |
| record_format | Article |
| series | Journal of Information Technology Management |
| spelling | doaj-art-573b5feb6aeb425da2215a72aa4d5e8b2025-08-20T03:10:38ZengUniversity of TehranJournal of Information Technology Management2008-58932423-50592020-12-011249010410.22059/jitm.2020.296305.245578401A Novel Scheme for Improving Accuracy of KNN Classification Algorithm Based on the New Weighting Technique and Stepwise Feature SelectionSaeid Sheikhi0Mohammad Taghi Kheirabadi1Amin Bazzazi2MSc, Department of Computer, Gorgan Branch, Islamic Azad University, Gorgan, Iran.Assistant Prof., Department of Computer, Gorgan Branch, Islamic Azad University, Gorgan, Iran.Assistant Prof., Department of Computer, Gorgan Branch, Islamic Azad University, Gorgan, Iran.K nearest neighbor algorithm is one of the most frequently used techniques in data mining for its integrity and performance. Though the KNN algorithm is highly effective in many cases, it has some essential deficiencies, which affects the classification accuracy of the algorithm. First, the effectiveness of the algorithm is affected by redundant and irrelevant features. Furthermore, this algorithm does not consider the differences between samples, which led the algorithm to have inaccurate predictions. In this paper, we proposed a novel scheme for improving the accuracy of the KNN classification algorithm based on the new weighting technique and stepwise feature selection. First, we used a stepwise feature selection method to eliminate irrelevant features and select highly correlated features with the class category. Then a new weighting method was proposed to give authority value to each sample in train dataset based on neighbor categories and Euclidean distances. This weighting approach gives a higher preference to samples that have neighbors with close Euclidean distance while they are in the same category, which can effectively increase the classification accuracy of the algorithm. We evaluated the accuracy rate of the proposed method and analyzed it with the traditional KNN algorithm and some similar works with the use of five real-world UCI datasets. The experiment results determined that the proposed scheme (denoted by WAD-KNN) performed better than the traditional KNN algorithm and considered approaches with the improvement of approximately 10% accuracy.https://jitm.ut.ac.ir/article_78401_aa4ee9377b284f7eaf141c5a92adbe96.pdfdata miningknn algorithmclassification algorithmweighted knn |
| spellingShingle | Saeid Sheikhi Mohammad Taghi Kheirabadi Amin Bazzazi A Novel Scheme for Improving Accuracy of KNN Classification Algorithm Based on the New Weighting Technique and Stepwise Feature Selection Journal of Information Technology Management data mining knn algorithm classification algorithm weighted knn |
| title | A Novel Scheme for Improving Accuracy of KNN Classification Algorithm Based on the New Weighting Technique and Stepwise Feature Selection |
| title_full | A Novel Scheme for Improving Accuracy of KNN Classification Algorithm Based on the New Weighting Technique and Stepwise Feature Selection |
| title_fullStr | A Novel Scheme for Improving Accuracy of KNN Classification Algorithm Based on the New Weighting Technique and Stepwise Feature Selection |
| title_full_unstemmed | A Novel Scheme for Improving Accuracy of KNN Classification Algorithm Based on the New Weighting Technique and Stepwise Feature Selection |
| title_short | A Novel Scheme for Improving Accuracy of KNN Classification Algorithm Based on the New Weighting Technique and Stepwise Feature Selection |
| title_sort | novel scheme for improving accuracy of knn classification algorithm based on the new weighting technique and stepwise feature selection |
| topic | data mining knn algorithm classification algorithm weighted knn |
| url | https://jitm.ut.ac.ir/article_78401_aa4ee9377b284f7eaf141c5a92adbe96.pdf |
| work_keys_str_mv | AT saeidsheikhi anovelschemeforimprovingaccuracyofknnclassificationalgorithmbasedonthenewweightingtechniqueandstepwisefeatureselection AT mohammadtaghikheirabadi anovelschemeforimprovingaccuracyofknnclassificationalgorithmbasedonthenewweightingtechniqueandstepwisefeatureselection AT aminbazzazi anovelschemeforimprovingaccuracyofknnclassificationalgorithmbasedonthenewweightingtechniqueandstepwisefeatureselection AT saeidsheikhi novelschemeforimprovingaccuracyofknnclassificationalgorithmbasedonthenewweightingtechniqueandstepwisefeatureselection AT mohammadtaghikheirabadi novelschemeforimprovingaccuracyofknnclassificationalgorithmbasedonthenewweightingtechniqueandstepwisefeatureselection AT aminbazzazi novelschemeforimprovingaccuracyofknnclassificationalgorithmbasedonthenewweightingtechniqueandstepwisefeatureselection |