多模态产品型号知识融合下图纸与文本跨模态整合

打开文本图片集

中图分类号：TP391 文献标识码：A文章编号：1006-8228（2025）09-45-05

Cross-modal Integration of Drawings and Text under Multimodal Product Type Knowledge Fusion

LiuYan，Wu Yonghui

（Shanghai Aviation Industrial（Group）Co.，Ltd.，Shanghai 2oo126，China

Abstract：Whenstitchingthetextinthedrawingdomainundersingle-modalfeatures，thereisonlyshallowsemanticcorelation， whichmakesitdificulttominedeepsemanticconnectionsandreducesintegrationeficiencyTherefore，thispaperconducts researchoncros-modalintegrationmethodsofdrawingsandtextsinthecontextofmutimodalproducttypeknowledgefusion. Firstly，amulti-layerconvolutionalneuralnetworkisusedtopreprocessthedrawingdataandextractfeatures;secondlya multimodalsemanticspaceisconstructedbasedontheproducttypeknowledgegraph，andentitesextractedfromdrawingfeatures aremappedandalignedwithentitiesintextsemanticrepresentations;thirdlythecros-modalintegrationofdrawingsandtextsis realizedthroughadynamicatentionmechanism.Theexperimentalresultsshowthatthemaximumredundancyrateofthismethod when applied does not exceed 0.5% ，and the average integration speed reaches groups，effectively reducing redundant informationincro-modaldataandreducingresoureconsumption，whichcanmeetthereqirementsofindustrialscenarioswith high real-time performance and low resource consumption.

Keywords：Multimodal Product Type Knowledge Fusion； Drawing;Text; Cross-modal; Integration

0引言

在工业设计、建筑规划及医疗诊断等实际应用场景中，图纸与文本作为信息的重要载体，共同承载关键信息，但二者因模态差异，使得传统单模态分析方法在处理时面临诸多困境。（剩余6060字）