Ph. D. Project
Title:
Identifiability of self-supervised learning using low-rank tensor decomposition
Dates:
2026/02/04 - 2029/02/08
Student:
Supervisor(s):
Description:
Self-supervised learning can leverage unlabeled datasets in order to learn a nonlinear transformation of data (a representation) that is informative in other prediction tasks such as classification. However, despite the importance of this approach, understanding under which conditions the learned transformation is guaranteed to be stable and identifiable, which are key properties to guarantee the interpretability of such approaches, is still an open question. Important questions are the influence of small models and heterogeneous data, which are practical scenarios of particular interest in applications due to addressing the non-i.i.d. data setting and lowering environmental impact. This project aims to study under which the self-supervised representations learned from small and heterogeneous data are identifiable and stable by leveraging the theory of low-rank tensor decompositions, including the canonical polyadic decomposition (CPD), the X-rank decomposition, and emerging non-additive decompositions such as the ParaTuck. These decompositions are algebraic models that possess strong uniqueness guarantees and can supply the essential mathematical framework to this project.
References:
T. G. Kolda, and B. W. Bader. Tensor decompositions and applications. SIAM review, 51(3), 455-500, 2009.
X. Liu, F. Zhang, Z. Hou, L. Mian, Z. Wang, J. Zhang, and J. Tang, "Self-supervised learning: Generative or contrastive," IEEE transactions on knowledge and data engineering, vol. 35, no. 1, pp. 857⬓876, 2021.
R. A. Borsoi, K. Usevich, D. Brie, and T. Adali, "Personalized coupled tensor decomposition for multimodal data fusion: Uniqueness and algorithms," IEEE Transactions on Signal Processing, 2024.
A. Cichocki, A.-H. Phan, Q. Zhao, N. Lee, I. Oseledets, et al., "Tensor networks for dimensionality reduction and large-scale optimization: Part 2 applications and future perspectives," Foundations and Trends in Machine Learning, vol. 9, no. 6, pp. 431⬓673, 2017.
References:
T. G. Kolda, and B. W. Bader. Tensor decompositions and applications. SIAM review, 51(3), 455-500, 2009.
X. Liu, F. Zhang, Z. Hou, L. Mian, Z. Wang, J. Zhang, and J. Tang, "Self-supervised learning: Generative or contrastive," IEEE transactions on knowledge and data engineering, vol. 35, no. 1, pp. 857⬓876, 2021.
R. A. Borsoi, K. Usevich, D. Brie, and T. Adali, "Personalized coupled tensor decomposition for multimodal data fusion: Uniqueness and algorithms," IEEE Transactions on Signal Processing, 2024.
A. Cichocki, A.-H. Phan, Q. Zhao, N. Lee, I. Oseledets, et al., "Tensor networks for dimensionality reduction and large-scale optimization: Part 2 applications and future perspectives," Foundations and Trends in Machine Learning, vol. 9, no. 6, pp. 431⬓673, 2017.
Keywords:
Identifiability, low-rank tensor decompositions, self-supervised learning
Department(s):
| Biology, Signals and Systems in Cancer and Neuroscience |