Enhancing cross entropy with a linearly adaptive loss function for optimized classification performance
Abstract We propose the linearly adaptive cross entropy loss function. This is a novel measure derived from the information theory. In comparison to the standard cross entropy loss function, the proposed one has an additional term that depends on the predicted probability of the true class. This fea...
Saved in:
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2024-11-01
|
| Series: | Scientific Reports |
| Subjects: | |
| Online Access: | https://doi.org/10.1038/s41598-024-78858-6 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850196233677701120 |
|---|---|
| author | Jae Wan Shim |
| author_facet | Jae Wan Shim |
| author_sort | Jae Wan Shim |
| collection | DOAJ |
| description | Abstract We propose the linearly adaptive cross entropy loss function. This is a novel measure derived from the information theory. In comparison to the standard cross entropy loss function, the proposed one has an additional term that depends on the predicted probability of the true class. This feature serves to enhance the optimization process in classification tasks involving one-hot encoded class labels. The proposed one has been evaluated on a ResNet-based model using the CIFAR-100 dataset. Preliminary results show that the proposed one consistently outperforms the standard cross entropy loss function in terms of classification accuracy. Moreover, the proposed one maintains simplicity, achieving practically the same efficiency to the traditional cross entropy loss. These findings suggest that our approach could broaden the scope for future research into loss function design. |
| format | Article |
| id | doaj-art-e59e48b938d641ba8bd7b32df02f30a8 |
| institution | OA Journals |
| issn | 2045-2322 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-e59e48b938d641ba8bd7b32df02f30a82025-08-20T02:13:31ZengNature PortfolioScientific Reports2045-23222024-11-011411610.1038/s41598-024-78858-6Enhancing cross entropy with a linearly adaptive loss function for optimized classification performanceJae Wan Shim0Extreme Materials Research Center, Korea Institute of Science and TechnologyAbstract We propose the linearly adaptive cross entropy loss function. This is a novel measure derived from the information theory. In comparison to the standard cross entropy loss function, the proposed one has an additional term that depends on the predicted probability of the true class. This feature serves to enhance the optimization process in classification tasks involving one-hot encoded class labels. The proposed one has been evaluated on a ResNet-based model using the CIFAR-100 dataset. Preliminary results show that the proposed one consistently outperforms the standard cross entropy loss function in terms of classification accuracy. Moreover, the proposed one maintains simplicity, achieving practically the same efficiency to the traditional cross entropy loss. These findings suggest that our approach could broaden the scope for future research into loss function design.https://doi.org/10.1038/s41598-024-78858-6Loss functionCross entropyLinearly adaptive lossCost function |
| spellingShingle | Jae Wan Shim Enhancing cross entropy with a linearly adaptive loss function for optimized classification performance Scientific Reports Loss function Cross entropy Linearly adaptive loss Cost function |
| title | Enhancing cross entropy with a linearly adaptive loss function for optimized classification performance |
| title_full | Enhancing cross entropy with a linearly adaptive loss function for optimized classification performance |
| title_fullStr | Enhancing cross entropy with a linearly adaptive loss function for optimized classification performance |
| title_full_unstemmed | Enhancing cross entropy with a linearly adaptive loss function for optimized classification performance |
| title_short | Enhancing cross entropy with a linearly adaptive loss function for optimized classification performance |
| title_sort | enhancing cross entropy with a linearly adaptive loss function for optimized classification performance |
| topic | Loss function Cross entropy Linearly adaptive loss Cost function |
| url | https://doi.org/10.1038/s41598-024-78858-6 |
| work_keys_str_mv | AT jaewanshim enhancingcrossentropywithalinearlyadaptivelossfunctionforoptimizedclassificationperformance |