Deep Neural Framework With Visual Attention and Global Context for Predicting Image Aesthetics
Computational inference of aesthetics has recently become a hot topic due to its usefulness in widely applications such as evaluating image quality, retouching image and retrieving image. Owing to the subjectivity of this problem, there is no general framework to predict image aesthetics. In this pa...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/9162034/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849714616284741632 |
|---|---|
| author | Yifei Xu Nuo Zhang Pingping Wei Genan Sang Li Li Feng Yuan |
| author_facet | Yifei Xu Nuo Zhang Pingping Wei Genan Sang Li Li Feng Yuan |
| author_sort | Yifei Xu |
| collection | DOAJ |
| description | Computational inference of aesthetics has recently become a hot topic due to its usefulness in widely applications such as evaluating image quality, retouching image and retrieving image. Owing to the subjectivity of this problem, there is no general framework to predict image aesthetics. In this paper, we propose a deep neural framework with visual attention module, self-generated global features and hybrid loss to address this problem. Specifically, the framework can be any state-of-the-art convolution classification network compatible with visual attention. Further, self-generated global feature compensates for the loss of global context information during training stage, and the hybrid loss guides the network to learn the similarity between the predicted aesthetic scores and the ground-truths through fusing soft-max-entropy and Earth Mover’s Distance(EMD). With the above-mentioned improvements, the proposed deep neural framework is capable of effectively predicting image aesthetics in an efficient way. In our experiments, we release a real-world aesthetic dataset that contains 1,800 2K photos labeled by several experienced photographers, and then provide a thorough ablation study of the design choices to better understand the superiority brought by each part of our framework, and design several comparisons with the state-of-the-art methods on a fraction of metrics. The experimental results on two datasets demonstrate that both accuracy and efficiency achieve favorably performance. |
| format | Article |
| id | doaj-art-03bf79aadace45919a9c11b7108db91f |
| institution | DOAJ |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-03bf79aadace45919a9c11b7108db91f2025-08-20T03:13:39ZengIEEEIEEE Access2169-35362025-01-011312402512403610.1109/ACCESS.2020.30150609162034Deep Neural Framework With Visual Attention and Global Context for Predicting Image AestheticsYifei Xu0https://orcid.org/0000-0003-3111-2518Nuo Zhang1Pingping Wei2Genan Sang3Li Li4Feng Yuan5School of Software, Xi’an Jiaotong University, Xi’an, ChinaSchool of Software, Xi’an Jiaotong University, Xi’an, ChinaState Key Laboratory for Manufacturing Systems Engineering, Xi’an Jiaotong University, Xi’an, ChinaAlltuu Inc., Hangzhou, ChinaAlltuu Inc., Hangzhou, ChinaAlltuu Inc., Hangzhou, ChinaComputational inference of aesthetics has recently become a hot topic due to its usefulness in widely applications such as evaluating image quality, retouching image and retrieving image. Owing to the subjectivity of this problem, there is no general framework to predict image aesthetics. In this paper, we propose a deep neural framework with visual attention module, self-generated global features and hybrid loss to address this problem. Specifically, the framework can be any state-of-the-art convolution classification network compatible with visual attention. Further, self-generated global feature compensates for the loss of global context information during training stage, and the hybrid loss guides the network to learn the similarity between the predicted aesthetic scores and the ground-truths through fusing soft-max-entropy and Earth Mover’s Distance(EMD). With the above-mentioned improvements, the proposed deep neural framework is capable of effectively predicting image aesthetics in an efficient way. In our experiments, we release a real-world aesthetic dataset that contains 1,800 2K photos labeled by several experienced photographers, and then provide a thorough ablation study of the design choices to better understand the superiority brought by each part of our framework, and design several comparisons with the state-of-the-art methods on a fraction of metrics. The experimental results on two datasets demonstrate that both accuracy and efficiency achieve favorably performance.https://ieeexplore.ieee.org/document/9162034/Image aestheticshybrid lossdeep neuralvisual attention |
| spellingShingle | Yifei Xu Nuo Zhang Pingping Wei Genan Sang Li Li Feng Yuan Deep Neural Framework With Visual Attention and Global Context for Predicting Image Aesthetics IEEE Access Image aesthetics hybrid loss deep neural visual attention |
| title | Deep Neural Framework With Visual Attention and Global Context for Predicting Image Aesthetics |
| title_full | Deep Neural Framework With Visual Attention and Global Context for Predicting Image Aesthetics |
| title_fullStr | Deep Neural Framework With Visual Attention and Global Context for Predicting Image Aesthetics |
| title_full_unstemmed | Deep Neural Framework With Visual Attention and Global Context for Predicting Image Aesthetics |
| title_short | Deep Neural Framework With Visual Attention and Global Context for Predicting Image Aesthetics |
| title_sort | deep neural framework with visual attention and global context for predicting image aesthetics |
| topic | Image aesthetics hybrid loss deep neural visual attention |
| url | https://ieeexplore.ieee.org/document/9162034/ |
| work_keys_str_mv | AT yifeixu deepneuralframeworkwithvisualattentionandglobalcontextforpredictingimageaesthetics AT nuozhang deepneuralframeworkwithvisualattentionandglobalcontextforpredictingimageaesthetics AT pingpingwei deepneuralframeworkwithvisualattentionandglobalcontextforpredictingimageaesthetics AT genansang deepneuralframeworkwithvisualattentionandglobalcontextforpredictingimageaesthetics AT lili deepneuralframeworkwithvisualattentionandglobalcontextforpredictingimageaesthetics AT fengyuan deepneuralframeworkwithvisualattentionandglobalcontextforpredictingimageaesthetics |