IoT-based approach to multimodal music emotion recognition
With the rapid development of Internet of Things (IoT) technology, multimodal emotion recognition has gradually become an important research direction in the field of artificial intelligence. However, existing methods often face challenges in efficiency and accuracy when processing multimodal data....
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2025-02-01
|
| Series: | Alexandria Engineering Journal |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S1110016824012158 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1825206936860098560 |
|---|---|
| author | Hanbing Zhao Ling Jin |
| author_facet | Hanbing Zhao Ling Jin |
| author_sort | Hanbing Zhao |
| collection | DOAJ |
| description | With the rapid development of Internet of Things (IoT) technology, multimodal emotion recognition has gradually become an important research direction in the field of artificial intelligence. However, existing methods often face challenges in efficiency and accuracy when processing multimodal data. This study aims to propose an IoT-supported multimodal music emotion recognition model that integrates audio and video signals to achieve real-time emotion recognition and classification. The proposed CGF-Net model combines a 3D Convolutional Neural Network (3D-CNN), Gated Recurrent Unit (GRU), and Fully Connected Network (FCN). By effectively fusing multimodal data, the model enhances the accuracy and efficiency of music emotion recognition. Extensive experiments were conducted on two public datasets, DEAM and DEAP, and the results demonstrate that CGF-Net performs exceptionally well in various emotion recognition tasks, particularly achieving high accuracy and F1 scores in recognizing positive emotions such as ”Happy” and ”Relax.” Compared to other benchmark models, CGF-Net shows significant advantages in both accuracy and stability. This study presents an effective solution for multimodal emotion recognition, demonstrating its broad potential in applications such as intelligent emotional interaction and music recommendation systems. |
| format | Article |
| id | doaj-art-38848c39588d4d66ab2720dc49182b02 |
| institution | Kabale University |
| issn | 1110-0168 |
| language | English |
| publishDate | 2025-02-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Alexandria Engineering Journal |
| spelling | doaj-art-38848c39588d4d66ab2720dc49182b022025-02-07T04:46:55ZengElsevierAlexandria Engineering Journal1110-01682025-02-011131931IoT-based approach to multimodal music emotion recognitionHanbing Zhao0Ling Jin1Department of Music Theory and Composition, College of Music, Beihua University, Jilin, 132100, ChinaDoctor of Bayung-playing, School of Music, Northeast Normal University, Jilin, 132100, China; Gnehesin Music Institute, Russia; Corresponding author at: Doctor of Bayung-playing, School of Music, Northeast Normal University, Jilin, 132100, China.With the rapid development of Internet of Things (IoT) technology, multimodal emotion recognition has gradually become an important research direction in the field of artificial intelligence. However, existing methods often face challenges in efficiency and accuracy when processing multimodal data. This study aims to propose an IoT-supported multimodal music emotion recognition model that integrates audio and video signals to achieve real-time emotion recognition and classification. The proposed CGF-Net model combines a 3D Convolutional Neural Network (3D-CNN), Gated Recurrent Unit (GRU), and Fully Connected Network (FCN). By effectively fusing multimodal data, the model enhances the accuracy and efficiency of music emotion recognition. Extensive experiments were conducted on two public datasets, DEAM and DEAP, and the results demonstrate that CGF-Net performs exceptionally well in various emotion recognition tasks, particularly achieving high accuracy and F1 scores in recognizing positive emotions such as ”Happy” and ”Relax.” Compared to other benchmark models, CGF-Net shows significant advantages in both accuracy and stability. This study presents an effective solution for multimodal emotion recognition, demonstrating its broad potential in applications such as intelligent emotional interaction and music recommendation systems.http://www.sciencedirect.com/science/article/pii/S1110016824012158IoTEmotion recognition technologyMusic analysisDeep learningMultimodal data fusion |
| spellingShingle | Hanbing Zhao Ling Jin IoT-based approach to multimodal music emotion recognition Alexandria Engineering Journal IoT Emotion recognition technology Music analysis Deep learning Multimodal data fusion |
| title | IoT-based approach to multimodal music emotion recognition |
| title_full | IoT-based approach to multimodal music emotion recognition |
| title_fullStr | IoT-based approach to multimodal music emotion recognition |
| title_full_unstemmed | IoT-based approach to multimodal music emotion recognition |
| title_short | IoT-based approach to multimodal music emotion recognition |
| title_sort | iot based approach to multimodal music emotion recognition |
| topic | IoT Emotion recognition technology Music analysis Deep learning Multimodal data fusion |
| url | http://www.sciencedirect.com/science/article/pii/S1110016824012158 |
| work_keys_str_mv | AT hanbingzhao iotbasedapproachtomultimodalmusicemotionrecognition AT lingjin iotbasedapproachtomultimodalmusicemotionrecognition |