Ph. D. Project
Title:
Multimodal data fusion with applications to geological imaging
Dates:
2025/11/19 - 2028/11/18
Student:
Supervisor(s):
Other supervisor(s):
DELCHINI Sylvain (s.delchini@brgm.fr)
Description:
In several fields, ranging from health to geosciences, information associated with the same phenomenon can be acquired using different detectors
under various acquisition conditions. Using a single acquisition is often insufficient to obtain a complete understanding of the observed phenomenon
[1]. The joint use of multimodal observations requires an information fusion step. The latter can be performed directly on the raw data from the
different modalities or on features extracted from the data.
This thesis project is part of the Sous-Sol Innovtech PEPR. One of the objectives of this project is to design methods for the mineralogical
characterization of geological samples using data from several spectroscopy modalities, including X-ray fluorescence, Raman spectroscopy, and
infrared spectroscopy. Identifying minerals sometimes requires the use of several acquisition modalities corresponding to different spectral bands
such as near infrared (NIR), shortwave infrared (SWIR), and midwave infrared (MWIR). The data from these different modalities must be mapped to
a common digital representation in order to be used. The objective of the thesis is to propose fast fusion algorithms for processing images containing
up to 10^5 voxels, each consisting of around 100 wavelengths. As a first step, the focus will be on fusion methods based on optimal transport [2,3]
and low-rank tensor decompositions [4,5]. Subsequently, the work will focus on deep learning-based methods. The proposed algorithms will be
assessed using real data on a BRGM platform in Orléans.
[1] D. Lahat, T. Adali, C. Jutten. "Multimodal Data Fusion: An Overview of Methods, Challenges and Prospects," Proceedings of the IEEE, vol. 3,
no. 9, 2015.
[2] N. Courty, R. Flamary, D. Tuia, T. Corpetti. "Optimal Transport for Data Fusion in Remote Sensing," IGARSS, Jul 2016, Beijing, China.
[3] J. Mifdal, B. Coll, N. Courty, J. Froment, B. Vedel. "Hyperspectral and multispectral Wasserstein barycenter for image fusion," IGARSS 2017,
Jul 2017, Houston, United States.
[4] C. I. Kanatsoulis, X. Fu, N. D. Sidiropoulos, W.-K. Ma, Hyperspectral super-resolution: A coupled tensor factorization approach," IEEE Trans.
Signal Process., vol. 66, no. 24, pp. 6503⬓6517, Dec. 2018.
[5] C. Prévost, K. Usevich, D. Brie, P. Comon. "Hyperspectral Super-Resolution With Coupled Tucker Approximation: Recoverability and SVD-
Based Algorithms," IEEE Trans. Signal Process., vol. 68, 2020.
under various acquisition conditions. Using a single acquisition is often insufficient to obtain a complete understanding of the observed phenomenon
[1]. The joint use of multimodal observations requires an information fusion step. The latter can be performed directly on the raw data from the
different modalities or on features extracted from the data.
This thesis project is part of the Sous-Sol Innovtech PEPR. One of the objectives of this project is to design methods for the mineralogical
characterization of geological samples using data from several spectroscopy modalities, including X-ray fluorescence, Raman spectroscopy, and
infrared spectroscopy. Identifying minerals sometimes requires the use of several acquisition modalities corresponding to different spectral bands
such as near infrared (NIR), shortwave infrared (SWIR), and midwave infrared (MWIR). The data from these different modalities must be mapped to
a common digital representation in order to be used. The objective of the thesis is to propose fast fusion algorithms for processing images containing
up to 10^5 voxels, each consisting of around 100 wavelengths. As a first step, the focus will be on fusion methods based on optimal transport [2,3]
and low-rank tensor decompositions [4,5]. Subsequently, the work will focus on deep learning-based methods. The proposed algorithms will be
assessed using real data on a BRGM platform in Orléans.
[1] D. Lahat, T. Adali, C. Jutten. "Multimodal Data Fusion: An Overview of Methods, Challenges and Prospects," Proceedings of the IEEE, vol. 3,
no. 9, 2015.
[2] N. Courty, R. Flamary, D. Tuia, T. Corpetti. "Optimal Transport for Data Fusion in Remote Sensing," IGARSS, Jul 2016, Beijing, China.
[3] J. Mifdal, B. Coll, N. Courty, J. Froment, B. Vedel. "Hyperspectral and multispectral Wasserstein barycenter for image fusion," IGARSS 2017,
Jul 2017, Houston, United States.
[4] C. I. Kanatsoulis, X. Fu, N. D. Sidiropoulos, W.-K. Ma, Hyperspectral super-resolution: A coupled tensor factorization approach," IEEE Trans.
Signal Process., vol. 66, no. 24, pp. 6503⬓6517, Dec. 2018.
[5] C. Prévost, K. Usevich, D. Brie, P. Comon. "Hyperspectral Super-Resolution With Coupled Tucker Approximation: Recoverability and SVD-
Based Algorithms," IEEE Trans. Signal Process., vol. 68, 2020.
Keywords:
Hyperspectral image, data fusion, optimal transport, tensor decomposition, low-rank approximation
Department(s):
| Biology, Signals and Systems in Cancer and Neuroscience |