Streaming Bayesian inference: theoretical limits and mini-batch approximate message-passing - CEA - Commissariat à l’énergie atomique et aux énergies alternatives Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2017

Streaming Bayesian inference: theoretical limits and mini-batch approximate message-passing

Résumé

In statistical learning for real-world large-scale data problems, one must often resort to "streaming" algorithms which operate sequentially on small batches of data. In this work, we present an analysis of the information-theoretic limits of mini-batch inference in the context of generalized linear models and low-rank matrix factorization. In a controlled Bayes-optimal setting, we characterize the optimal performance and phase transitions as a function of mini-batch size. We base part of our results on a detailed analysis of a mini-batch version of the approximate message-passing algorithm (Mini-AMP), which we introduce. Additionally, we show that this theoretical optimality carries over into real-data problems by illustrating that Mini-AMP is competitive with standard streaming algorithms for clustering.
Fichier principal
Vignette du fichier
1706.00705.pdf (1.2 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

cea-01553517 , version 1 (03-07-2017)

Identifiants

Citer

Andre Manoel, Florent Krzakala, Eric W. Tramel, Lenka Zdeborová. Streaming Bayesian inference: theoretical limits and mini-batch approximate message-passing. 2017. ⟨cea-01553517⟩
389 Consultations
337 Téléchargements

Altmetric

Partager

Gmail Mastodon Facebook X LinkedIn More