When neural networks using different sensors create similar features
Résumé
Multimodal problems are omnipresent in the real world: autonomous driving, robotic grasping, scene understanding, etc... Instead of proposing to improve an existing method or algorithm: we will use existing statistical methods to understand the features in already-existing neural networks. More precisely, we demonstrate that a fusion method relying on Canonical Correlation Analysis on features extracted from Deep Neural Networks using different sensors is equivalent to looking at the output of the networks themselves.