Search

Abstract In recent years, Artificial Intelligence Generated Content (AIGC) technologies have advanced rapidly, with models such as Stable Diffusion and GPT garnering significant attention across various domains. Against this backdrop, AI-driven music composition techniques have also produced significant progress. However, no existing model has yet demonstrated the capability to generate Chinese-style music corresponding to Chinese-style videos. To address this gap, this study proposes a novel Chinese-style video music generation model based on the Latent Diffusion Model (LDM) and Diffusion Transformers (DiT). Experimental results demonstrate that the proposed model generates Chinese-style music from Chinese-style videos and achieves performance comparable to the baseline models in audio quality, distribution fitting, musicality, rhythmic stability, and audio-visual synchronization. These findings indicate that the model captures the stylistic features of Chinese music. This research not only demonstrates the feasibility applications of artificial intelligence in music creation but also provides a new technological approach to preserve and innovate the traditional Chinese music culture in the digital era. Furthermore, it explores new possibilities for the dissemination and innovation of Chinese cultural arts in the digital age.

In the context of rapid advancements in artificial intelligence technology, AI-powered music composition has demonstrated remarkable creative capabilities. However, no existing music generation model has been able to produce authentic waveform-level traditional Chinese music. To explore the potential of this field and address the limitations of current technologies in generating traditional Chinese music, this study introduces CPTGZ (Chinese Painting to Guzheng Music), a music generation model based on latent diffusion and Transformer architectures. CPTGZ aims to achieve automatic generation of waveform-level Guzheng music from Chinese paintings, thereby addressing the inability of existing music generation models to produce traditional Chinese music.To support the development and training of the model, we constructed a large-scale dataset of paired Chinese paintings and Guzheng music, consisting of 22,103 sample pairs. Through experimental evaluation, we found that CPTGZ exhibits excellent performance in terms of music quality and Guzheng-specific characteristics. The results demonstrate that our model can generate Chinese Guzheng music pieces highly correlated in style and semantics with the input Chinese paintings. Furthermore, the musical qualities of the generated Guzheng compositions demonstrate the characteristics of traditional Chinese music, thus validating the feasibility and effectiveness of our model.This research contributes to the field of AI-driven music generation by addressing the specific challenges of creating authentic traditional Chinese music, particularly Guzheng compositions, based on visual art inputs. The successful implementation of CPTGZ not only opens new avenues for cross-modal generation in the domain of culturally specific art forms, but also demonstrates the potential for AI to preserve and innovate within traditional art forms.