Subject Training

The recent advent of multimodal large language models (or more generally, foundation models) such as GPT-3--GPT-5 (OpenAI) or LLaMA (Meta), raises the issue of the explosion in the number of their parameters (more than 10¹² parameters for recent versions of GPT). This implies a significant energy cost for training and inference, as well as a large storage volume for the network weights in particular, making their deployment on devices such as mobile phones difficult or even impossible. The project falls within the so-called theme of Frugal Artificial Intelligence, which aims to propose architectures with reduced computational footprint for their training, inference and storage. The project aims at combining insights from low-rank tensor decompositions, information theory and federated learning to achieve highly efficient model compression.