CRAKUT：融合对比区域注意力机制与临床先验知识的U-Transformer用于放射学报告生成

打开文本图片集

Abstract：ObjectiveWeproposeaContrastiveRegionalAtentionandPriorKnowledge-InfusedU-Transformermodel （CRAKUT）toaddresstlengesofimbalancedtextdistrbtio，ackofoextalcliicalowledgeandosl informationtransformationtoenhancethequalityof generatedradiologyreports.MethodsTheCRAKUTmodelcomprises3 keycomponents，includinganimageencoder thatutilizescommonnormalimagesfromthedatasetforextractingenhanced visualfeaturesetealoedgeiusertaticopoatesiicalpriorowledgendaU-aforertatailitate cros-modal informationconversion from vision tolanguage.Thecontrastive regional atentioninthe imageencoder was introducedtoenhancethefeaturesofabnormalregionsbyemphasizingthediferencebetweennormalandabnormalsemantic features.Additionall，theclinical prior knowledge infuser within the textencoder integratesclinical history and knowledge graphs generatedby ChatGPT.Finaly，the U-Transformer was utilizedtoconnect the multi-modal encoderandtheeport decoderinaU-connectionschema，andmultipletypesofinformationwereusedtofuseandotainthefinalreport.Results We evaluated theproposedCRAKUTmodelon twopubliclyavailable CXR datasets （IU-XrayandMIMIC- .CXR′ ）.Theexperimental results showed that theCRAKUTmodelachieved astate-of-the-art performanceonreport generationwithaBLEU-4scoreof 0.159，aROUGE-Lscoreof0.353，andaCDErscoreof050inMC-CXRdataset;themodelalsohadaMETEORscoreof 0.258inIU-Xraydataset，outperformingallthecomparisonmodels.ConclusionTheproposedmethodhasgreatpotentialfor application inclinical disease diagnosesand report generation.

Keywords： ChestX-ray;contrastiveregionattention;clinical priorknowledge;cross-modal; U-Transformermodel

胸片是临床实践中的基本诊断工具，广泛用于各种呼吸和心脏疾病的早期检测、诊断和管理。（剩余17496字）