CPTGZ: Generating Chinese Guzheng Music From Chinese Paintings Based on Diffusion Model
In the context of rapid advancements in artificial intelligence technology, AI-powered music composition has demonstrated remarkable creative capabilities. However, no existing music generation model has been able to produce authentic waveform-level traditional Chinese music. To explore the potentia...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10711246/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1846162879531712512 |
|---|---|
| author | Enji Zhao Jiaxiang Zheng Moxi Cao |
| author_facet | Enji Zhao Jiaxiang Zheng Moxi Cao |
| author_sort | Enji Zhao |
| collection | DOAJ |
| description | In the context of rapid advancements in artificial intelligence technology, AI-powered music composition has demonstrated remarkable creative capabilities. However, no existing music generation model has been able to produce authentic waveform-level traditional Chinese music. To explore the potential of this field and address the limitations of current technologies in generating traditional Chinese music, this study introduces CPTGZ (Chinese Painting to Guzheng Music), a music generation model based on latent diffusion and Transformer architectures. CPTGZ aims to achieve automatic generation of waveform-level Guzheng music from Chinese paintings, thereby addressing the inability of existing music generation models to produce traditional Chinese music.To support the development and training of the model, we constructed a large-scale dataset of paired Chinese paintings and Guzheng music, consisting of 22,103 sample pairs. Through experimental evaluation, we found that CPTGZ exhibits excellent performance in terms of music quality and Guzheng-specific characteristics. The results demonstrate that our model can generate Chinese Guzheng music pieces highly correlated in style and semantics with the input Chinese paintings. Furthermore, the musical qualities of the generated Guzheng compositions demonstrate the characteristics of traditional Chinese music, thus validating the feasibility and effectiveness of our model.This research contributes to the field of AI-driven music generation by addressing the specific challenges of creating authentic traditional Chinese music, particularly Guzheng compositions, based on visual art inputs. The successful implementation of CPTGZ not only opens new avenues for cross-modal generation in the domain of culturally specific art forms, but also demonstrates the potential for AI to preserve and innovate within traditional art forms. |
| format | Article |
| id | doaj-art-07f3ba106248461481073364b09feea8 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-07f3ba106248461481073364b09feea82024-11-20T00:01:18ZengIEEEIEEE Access2169-35362024-01-011216924716926210.1109/ACCESS.2024.347699810711246CPTGZ: Generating Chinese Guzheng Music From Chinese Paintings Based on Diffusion ModelEnji Zhao0https://orcid.org/0009-0004-9643-0757Jiaxiang Zheng1https://orcid.org/0009-0000-1407-8894Moxi Cao2https://orcid.org/0009-0000-2769-2316Department of Global Cultural Convergence, Graduate School, Kangwon National University, Chuncheon, Gangwon-do, South KoreaDepartment of Global Cultural Convergence, Graduate School, Kangwon National University, Chuncheon, Gangwon-do, South KoreaDepartment of Global Cultural Convergence, Graduate School, Kangwon National University, Chuncheon, Gangwon-do, South KoreaIn the context of rapid advancements in artificial intelligence technology, AI-powered music composition has demonstrated remarkable creative capabilities. However, no existing music generation model has been able to produce authentic waveform-level traditional Chinese music. To explore the potential of this field and address the limitations of current technologies in generating traditional Chinese music, this study introduces CPTGZ (Chinese Painting to Guzheng Music), a music generation model based on latent diffusion and Transformer architectures. CPTGZ aims to achieve automatic generation of waveform-level Guzheng music from Chinese paintings, thereby addressing the inability of existing music generation models to produce traditional Chinese music.To support the development and training of the model, we constructed a large-scale dataset of paired Chinese paintings and Guzheng music, consisting of 22,103 sample pairs. Through experimental evaluation, we found that CPTGZ exhibits excellent performance in terms of music quality and Guzheng-specific characteristics. The results demonstrate that our model can generate Chinese Guzheng music pieces highly correlated in style and semantics with the input Chinese paintings. Furthermore, the musical qualities of the generated Guzheng compositions demonstrate the characteristics of traditional Chinese music, thus validating the feasibility and effectiveness of our model.This research contributes to the field of AI-driven music generation by addressing the specific challenges of creating authentic traditional Chinese music, particularly Guzheng compositions, based on visual art inputs. The successful implementation of CPTGZ not only opens new avenues for cross-modal generation in the domain of culturally specific art forms, but also demonstrates the potential for AI to preserve and innovate within traditional art forms.https://ieeexplore.ieee.org/document/10711246/Music generationlatent diffusion modeltraditional Chinese musicdeep learningAI music composition |
| spellingShingle | Enji Zhao Jiaxiang Zheng Moxi Cao CPTGZ: Generating Chinese Guzheng Music From Chinese Paintings Based on Diffusion Model IEEE Access Music generation latent diffusion model traditional Chinese music deep learning AI music composition |
| title | CPTGZ: Generating Chinese Guzheng Music From Chinese Paintings Based on Diffusion Model |
| title_full | CPTGZ: Generating Chinese Guzheng Music From Chinese Paintings Based on Diffusion Model |
| title_fullStr | CPTGZ: Generating Chinese Guzheng Music From Chinese Paintings Based on Diffusion Model |
| title_full_unstemmed | CPTGZ: Generating Chinese Guzheng Music From Chinese Paintings Based on Diffusion Model |
| title_short | CPTGZ: Generating Chinese Guzheng Music From Chinese Paintings Based on Diffusion Model |
| title_sort | cptgz generating chinese guzheng music from chinese paintings based on diffusion model |
| topic | Music generation latent diffusion model traditional Chinese music deep learning AI music composition |
| url | https://ieeexplore.ieee.org/document/10711246/ |
| work_keys_str_mv | AT enjizhao cptgzgeneratingchineseguzhengmusicfromchinesepaintingsbasedondiffusionmodel AT jiaxiangzheng cptgzgeneratingchineseguzhengmusicfromchinesepaintingsbasedondiffusionmodel AT moxicao cptgzgeneratingchineseguzhengmusicfromchinesepaintingsbasedondiffusionmodel |