基于多粒度渐进式融合的多模态命名实体识别方法

打开文本图片集

关键词：多模态命名实体识别；多模态融合；多粒度；渐进式融合；命名实体识别中图分类号：TP391 文献标志码：A 文章编号：1001-3695（2025）10-017-3027-07doi：10.19734/j. issn.1001-3695.2025.03.0071

Multimodal named entity recognition method based on multi-granularity progressive fusion

Ying Xujiana，Zhu Yanhuib†，Chen Haoʰ，Man Fangtengα，Zhang Zhixuanʰ （aSchoolofuefleliclspUest nan 412007，China）

Abstract：Toaddressthelackoffine-grainedsemanticsand inconsistent multimodalrepresentationsinexisting multimodal named entityrecognition methods，thispaperproposedamulti-granularityprogresivefusionapproach formultimodal named entityrecognition.Firstly，theproposedframework incorporatedadynamicgatedfilteringmechanismtoselectivelyextractvisual regionfeaturesrelevanttoextualsmanticsviaross-modal dynamicweighting.Additionaly，theframework integratedrossmodalalignmentandadversarialperturbationmechanisms tostrengthenconsistencyandgeneralizationbetweentextualfeatures andglobalvisualrepresentations.Secondlyitdesignedamulti-levelprogressivefusionnetworktoconstructanoisesuppressedandsemanticenhancedmulti-granularityrepresentationlearningsystem.Thisarchitecturehierarchicallintegrated text-level，text-regionmage-level，andext-globalimage-levefaturesthroughaparallelmulti-stagefusionsrategyfectielycombining hierarchical featurevectorsacross diferent granularities.ExtensiveexperimentsconductedontheTwiter-2015 and Twitter-2O17 benchmark datasets demonstrate that the proposed method achieves average F1 score improvements of 0.89% （204号 and 1.08% ，respectively，compared with other multimodal named entity recognition approaches，which confirms the model’s effectiveness in named entity recognition tasks.

Keywords：multimodal namedentityrecognition（MNER）；multimodalfusion；multi-granularity；progressivefusion；named entity recognition（NER）

0 引言

命名实体识别（NER）是自然语言处理（NLP）中的一个基本任务[1]，旨在从文本中识别并定位具有特定意义的实体，并将其分类到预定义的类别中。（剩余17274字）