XMolCap: Advancing Molecular Captioning Through Multimodal Fusion and Explainable Graph Neural Networks

  • Duong Thanh Tran
  • , Nguyen Doan Hieu Nguyen
  • , Nhat Truong Pham
  • , Rajan Rakkiyappan
  • , Rajendra Karki
  • , Balachandran Manavalan

Research output: Contribution to journalArticlepeer-review

Abstract

Large language models (LLMs) have significantly advanced computational biology by enabling the integration of molecular, protein, and natural language data to accelerate drug discovery. However, existing molecular captioning approaches often underutilize diverse molecular modalities and lack interpretability. In this study, we introduce XMolCap, a novel explainable molecular captioning framework that integrates molecular images, SMILES strings, and graph-based structures through a stacked multimodal fusion mechanism. The framework is built upon a BioT5-based encoder-decoder architecture, which serves as the backbone for extracting feature representations from SELFIES. By leveraging specialized models such as SwinOCSR, SciBERT, and GIN-MoMu, XMolCap effectively captures complementary information from each modality. Our model not only achieves state-of-the-art performance on two benchmark datasets (L+M-24 and ChEBI-20), outperforming several strong baselines, but also provides detailed, functional group-aware, and property-specific explanations through graph-based interpretation. XMolCap is publicly available at https://github.com/cbbl-skku-org/XMolCap/ for reproducibility and local deployment. We believe it holds strong potential for clinical and pharmaceutical applications by generating accurate, interpretable molecular descriptions that deepen our understanding of molecular properties and interactions.

Original languageEnglish
Pages (from-to)7034-7045
Number of pages12
JournalIEEE Journal of Biomedical and Health Informatics
Volume29
Issue number10
DOIs
StatePublished - 2025

Keywords

  • Explainable artificial intelligence
  • graph neural networks
  • language and molecules
  • large language models
  • model interpretation
  • molecular captioning
  • multimodal fusion

Fingerprint

Dive into the research topics of 'XMolCap: Advancing Molecular Captioning Through Multimodal Fusion and Explainable Graph Neural Networks'. Together they form a unique fingerprint.

Cite this