Re-Calibrating Network by Refining Initial Features Through Generative Gradient Regularization
In the domain of Deep Neural Networks (DNNs), the deployment of regularization techniques is a common strategy for optimizing network performance. While these methods have been shown to be effective for optimization, they typically necessitate complete retraining of the network. We propose a trainin...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10854681/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832575617668743168 |
|---|---|
| author | Naim Reza Ho Yub Jung |
| author_facet | Naim Reza Ho Yub Jung |
| author_sort | Naim Reza |
| collection | DOAJ |
| description | In the domain of Deep Neural Networks (DNNs), the deployment of regularization techniques is a common strategy for optimizing network performance. While these methods have been shown to be effective for optimization, they typically necessitate complete retraining of the network. We propose a training methodology that emphasizes on refining the features extracted from the initial layer of a DNN by regularizing the network with the help of a reference gradient. Our findings indicate that augmenting the gradients produced by the filters of the initial layer of a DNN, through the introduction of a reference gradient, leads to refined feature extraction and enhanced performance. We produce the reference gradient from the decoder of a generative network and subsequently encourage the target classifier network to adjust its weights to minimize discrepancies between the reference gradient and the gradient produced by the classifier network. The experiments show that implementing this method on a pre-trained network effectively re-calibrates the network and augments higher variance filters of the initial layer of the network, which helps produce refined features. Notably, this refinement in features translates to improved generalization and the proposed method also eliminates the necessity of total retraining of the target network. In empirical evaluation, we applied the proposed methodology to CIFAR, SVHN and ImageNet datasets, utilizing a range of network architectures. The results evidenced a performance gain of 1.66% for the CIFAR dataset using WideResNet, 1.22% for the SVHN dataset using PreResNet and 0.57% for the ImageNet dataset using ResNet. |
| format | Article |
| id | doaj-art-d0e8cd16dc18440bae1e743e12f3c341 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-d0e8cd16dc18440bae1e743e12f3c3412025-01-31T23:04:55ZengIEEEIEEE Access2169-35362025-01-0113201912020210.1109/ACCESS.2025.353421610854681Re-Calibrating Network by Refining Initial Features Through Generative Gradient RegularizationNaim Reza0https://orcid.org/0009-0007-0329-9369Ho Yub Jung1https://orcid.org/0000-0002-2906-9170Department of Computer Engineering, Chosun University, Gwangju, South KoreaDepartment of Computer Engineering, Chosun University, Gwangju, South KoreaIn the domain of Deep Neural Networks (DNNs), the deployment of regularization techniques is a common strategy for optimizing network performance. While these methods have been shown to be effective for optimization, they typically necessitate complete retraining of the network. We propose a training methodology that emphasizes on refining the features extracted from the initial layer of a DNN by regularizing the network with the help of a reference gradient. Our findings indicate that augmenting the gradients produced by the filters of the initial layer of a DNN, through the introduction of a reference gradient, leads to refined feature extraction and enhanced performance. We produce the reference gradient from the decoder of a generative network and subsequently encourage the target classifier network to adjust its weights to minimize discrepancies between the reference gradient and the gradient produced by the classifier network. The experiments show that implementing this method on a pre-trained network effectively re-calibrates the network and augments higher variance filters of the initial layer of the network, which helps produce refined features. Notably, this refinement in features translates to improved generalization and the proposed method also eliminates the necessity of total retraining of the target network. In empirical evaluation, we applied the proposed methodology to CIFAR, SVHN and ImageNet datasets, utilizing a range of network architectures. The results evidenced a performance gain of 1.66% for the CIFAR dataset using WideResNet, 1.22% for the SVHN dataset using PreResNet and 0.57% for the ImageNet dataset using ResNet.https://ieeexplore.ieee.org/document/10854681/Regularizationgradient regularizationre-calibrationdeep neural network |
| spellingShingle | Naim Reza Ho Yub Jung Re-Calibrating Network by Refining Initial Features Through Generative Gradient Regularization IEEE Access Regularization gradient regularization re-calibration deep neural network |
| title | Re-Calibrating Network by Refining Initial Features Through Generative Gradient Regularization |
| title_full | Re-Calibrating Network by Refining Initial Features Through Generative Gradient Regularization |
| title_fullStr | Re-Calibrating Network by Refining Initial Features Through Generative Gradient Regularization |
| title_full_unstemmed | Re-Calibrating Network by Refining Initial Features Through Generative Gradient Regularization |
| title_short | Re-Calibrating Network by Refining Initial Features Through Generative Gradient Regularization |
| title_sort | re calibrating network by refining initial features through generative gradient regularization |
| topic | Regularization gradient regularization re-calibration deep neural network |
| url | https://ieeexplore.ieee.org/document/10854681/ |
| work_keys_str_mv | AT naimreza recalibratingnetworkbyrefininginitialfeaturesthroughgenerativegradientregularization AT hoyubjung recalibratingnetworkbyrefininginitialfeaturesthroughgenerativegradientregularization |