cea-02524865
https://cea.hal.science/cea-02524865
https://cea.hal.science/cea-02524865/document
https://cea.hal.science/cea-02524865/file/publilfoini2.pdf
arxiv:1906.08479
doi:10.1088/1742-5468/ab4bbb
[CEA] CEA - Commissariat à l'énergie atomique
[UNIV-PARIS7] Université Denis Diderot - Paris VII
[ENS-PARIS] Ecole Normale Supérieure de Paris
[CNRS] CNRS - Centre national de la recherche scientifique
[OPENAIRE] OpenAIRE
[DSM-IPHT] IPHT
[DSV] Direction de la recherche fondamentale - sciences du vivant
[CEA-UPSAY] CEA - Université Paris-Saclay
[PSL] Université Paris sciences et lettres
[USPC] Université Sorbonne Paris Cité
[UNIV-PARIS-SACLAY] Université Paris-Saclay
[CEA-UPSAY-SACLAY] CEA-UPSAY-SACLAY
[CEA-DRF] Direction de Recherche Fondamentale
[SORBONNE-UNIVERSITE] Sorbonne Université
[SORBONNE-UNIV] Sorbonne Université 01/01/2018
[SU-SCIENCES] Faculté des Sciences de Sorbonne Université
[SU-SCI] Sciences - Sorbonne Université
[LPENS] Laboratoire de physique de l'ENS
[UNIV-PARIS] Université Paris Cité
[UP-SCIENCES] Université de Paris - Faculté des Sciences
[TEST-HALCNRS] Collection test HAL CNRS
[ENS-PSL] École normale supérieure - PSL
[SU-TI] Sorbonne Université - Texte Intégral
[ANR] ANR
[GS-MATHEMATIQUES] Graduate School Mathématiques
[GS-PHYSIQUE] Graduate School Physique
[ALLIANCE-SU] Alliance Sorbonne Université
High-temperature expansions and message passing algorithms
Maillard, Antoine
Foini, Laura
Castellanos, Alejandro Lage
Krzakala, Florent
Mezard, Marc
Zdeborová, Lenka
[PHYS] Physics [physics]
ART
random matrix theory and extensions
inference of graphical models
statistical inference
message-passing algorithms
Improved mean-eld technics are a central theme of statistical physics methods applied to inference and learning. We revisit here some of these methods using high-temperature expansions for disordered systems initiated by Plefka, Georges and Yedidia. We derive the Gibbs free entropy and the subsequent self-consistent equations for a generic class of statistical models with correlated matrices and show in particular that many classical approximation schemes, such as adaptive TAP, Expectation-Consistency, or the approximations behind the Vector Approximate Message Passing algorithm all rely on the same assumptions, that are also at the heart of high-temperature expansions. We focus on the case of rotationally invariant random coupling matrices in the 'high-dimensional' limit in which the number of samples and the dimension are both large, but with a xed ratio. This encapsulates many widely studied models, such as Restricted Boltzmann Machines or Generalized Linear Models with correlated data matrices. In this general setting, we show that all the approximation schemes described before are equivalent, and we conjecture that they are exact in the thermodynamic limit in the replica symmetric phases. We achieve this conclusion by resummation of the in nite perturbation series, which generalises a seminal result of Parisi and Potters. A rigorous derivation of this conjecture is an interesting mathematical challenge. On the way to these conclusions, we uncover several diagrammatical results in connection with free probability and random matrix theory, that are interesting independently of the rest of our work.
2019-11-01
2020-03-30
en
Journal of Statistical Mechanics: Theory and Experiment
IOP Publishing