Cross-modal prototype based multimodal federated learning under severely missing modality

Information Fusion; Volume 122, October 2025, 103219.

Guardat en:

Dades bibliogràfiques
Autors principals:	Le, Q. Huy, Thwal, Chu Myaet, Qiao, Yu, Tun, Ye Lin, Nguyen, Huu Nhat Minh, Huh, Eui-Nam, Hong, Choong Seon
Format:	Bài viết
Idioma:	English
Publicat:	Elsevier 2025
Matèries:	Multimodal federated learning (MFL) decentralized machine learning paradigm data heterogeneity autonomous driving
Accés en línia:	https://doi.org/10.1016/j.inffus.2025.103219 https://elib.vku.udn.vn/handle/123456789/5902
Etiquetes:	Afegir etiqueta Sense etiquetes, Sigues el primer a etiquetar aquest registre!

Thư viện lưu trữ:	Trường Đại học Công nghệ Thông tin và Truyền thông Việt Hàn - Đại học Đà Nẵng

id	oai:elib.vku.udn.vn:123456789-5902
record_format	dspace
spelling	oai:elib.vku.udn.vn:123456789-59022025-11-18T02:11:32Z Cross-modal prototype based multimodal federated learning under severely missing modality Le, Q. Huy Thwal, Chu Myaet Qiao, Yu Tun, Ye Lin Nguyen, Huu Nhat Minh Huh, Eui-Nam Hong, Choong Seon Multimodal federated learning (MFL) decentralized machine learning paradigm data heterogeneity autonomous driving Information Fusion; Volume 122, October 2025, 103219. Multimodal federated learning (MFL) has emerged as a decentralized machine learning paradigm, allowing multiple clients with different modalities to collaborate on training a global model across diverse data sources without sharing their private data. However, challenges, such as data heterogeneity and severely missing modalities, pose crucial hindrances to the robustness of MFL, significantly impacting the performance of global model. The occurrence of missing modalities in real-world applications, such as autonomous driving, often arises from factors like sensor failures, leading knowledge gaps during the training process. Specifically, the absence of a modality introduces misalignment during the local training phase, stemming from zero-filling in the case of clients with missing modalities. Consequently, achieving robust generalization in global model becomes imperative, especially when dealing with clients that have incomplete data. In this paper, we propose Multimodal Federated Cross Prototype Learning (MFCPL), a novel approach for MFL under severely missing modalities. Our MFCPL leverages the complete prototypes to provide diverse modality knowledge in modality-shared level with the cross-modal regularization and modality-specific level with cross-modal contrastive mechanism. Additionally, our approach introduces the cross-modal alignment to provide regularization for modality-specific features, thereby enhancing the overall performance, particularly in scenarios involving severely missing modalities. Through extensive experiments on four multimodal datasets, we demonstrate the effectiveness of MFCPL in mitigating the challenges of data heterogeneity and severely missing modalities while improving the overall performance and robustness of MFL. 2025-11-18T02:11:20Z 2025-11-18T02:11:20Z 2025-10 Working Paper https://doi.org/10.1016/j.inffus.2025.103219 https://elib.vku.udn.vn/handle/123456789/5902 en application/pdf Elsevier
institution	Trường Đại học Công nghệ Thông tin và Truyền thông Việt Hàn - Đại học Đà Nẵng
collection	DSpace
language	English
topic	Multimodal federated learning (MFL) decentralized machine learning paradigm data heterogeneity autonomous driving
spellingShingle	Multimodal federated learning (MFL) decentralized machine learning paradigm data heterogeneity autonomous driving Le, Q. Huy Thwal, Chu Myaet Qiao, Yu Tun, Ye Lin Nguyen, Huu Nhat Minh Huh, Eui-Nam Hong, Choong Seon Cross-modal prototype based multimodal federated learning under severely missing modality
description	Information Fusion; Volume 122, October 2025, 103219.
format	Working Paper
author	Le, Q. Huy Thwal, Chu Myaet Qiao, Yu Tun, Ye Lin Nguyen, Huu Nhat Minh Huh, Eui-Nam Hong, Choong Seon
author_facet	Le, Q. Huy Thwal, Chu Myaet Qiao, Yu Tun, Ye Lin Nguyen, Huu Nhat Minh Huh, Eui-Nam Hong, Choong Seon
author_sort	Le, Q. Huy
title	Cross-modal prototype based multimodal federated learning under severely missing modality
title_short	Cross-modal prototype based multimodal federated learning under severely missing modality
title_full	Cross-modal prototype based multimodal federated learning under severely missing modality
title_fullStr	Cross-modal prototype based multimodal federated learning under severely missing modality
title_full_unstemmed	Cross-modal prototype based multimodal federated learning under severely missing modality
title_sort	cross-modal prototype based multimodal federated learning under severely missing modality
publisher	Elsevier
publishDate	2025
url	https://doi.org/10.1016/j.inffus.2025.103219 https://elib.vku.udn.vn/handle/123456789/5902
_version_	1849386462390255616

Cross-modal prototype based multimodal federated learning under severely missing modality

Ítems similars