基于特性分流的多模态对话情绪感知算法

打开文本图片集

doi：10.19734/j. issn.1001-3695.2024.12.0466

Multimodal dialogue emotion perception algorithm based on feature divergence

Ren Qinze a，b ，Yuan Yea，b，Fu Ketinga，ʰ，Fu Junxiua，ʰ，Xu Kanga，b，Liu Na，bt （a.Institutefcneelie，ooflhe&，Ueitffone&oi ，China）

Abstract：Multimodalemotion perceptioniscrucialfor monitoring personal healthand providing medicalcareinthe fieldof proactive health.Currnt multimodal dialogue emotionperceptiontechnologiesfacechallenges ifusing informationacross differentmodalities，particularlyincapturinglocalrelationshipsbetweenmodalies.Theproposedmultimodalfusionalgorithm basedonfeaturediversion，MEPAD（multimodalemotionperceptionalgorithmwith featurediversion），addressedthesechallenges bycapturing global information indialogues using graphneural networks and integrating homogeneous and specific features across modalities through thehypercomplex number system and pairwise feature fusion mechanisms.Experiments on he IEMOCAP and MOSEI datasets demonstratethat MEPAD significantlyoutperforms existing methodsin multimodal dialogue emotionperceptiontasks，highlightingitsefectiveessandpotentialinhandlingomplexemotionaldata.Thisresearchoffs newinsights for theapplication of multimodal emotion perception technology in proactive health.

Key words：multimodal emotionrecognition；graph neural networks；hypercomplex number system；pairwisefeature fusion; dialogue emotion perception

0 引言

情绪感知是人类交流中的一个关键组成部分，在主动健康领域，对话者情绪变化也可以作为评估人类健康信息监控的重要指标[1]。（剩余18982字）