2025 International Conference on Sensing, Measurement & Data Analytics in the era of Artificial Intelligence (ICSMD 2025)

Multimodal learning often faces modality competition, where stronger modalities dominate and degrade overall performance. To address this, we propose the Cross-modal Competition Regularizer with Permutation - based Latent Regularizer (CCR-PLR), a game-theoretic information-driven framework that balances modality contributions. CCR models learning as a constant-sum game, encouraging each modality to enhance unique, task-relevant information while reducing redundancy. The PLR applies within-batch latent permutations and measures prediction shifts using Jensen - Shannon Divergence to approximate combinational mutual information efficiently. Combined with modality-specific and shared encoders, CCR-PLR explicitly optimizes complementary and discriminative representations. Experiments on a dual-modal pressure - vibration dataset show CCR-PLR consistently surpasses ResNet, Conformer, and CNN baselines in accuracy and robustness under corrupted inputs. Ablation results further verify that CCR-PLR effectively mitigates modality imbalance and improves generalization in cross-modal learning.

Fault Diagnosis