Preprints, Working Papers, ... Year : 2025

Refined Analysis of Federated Averaging's Bias and Federated Richardson-Romberg Extrapolation

Abstract

In this paper, we present a novel analysis of FedAvg with constant step size, relying on the Markov property of the underlying process. We demonstrate that the global iterates of the algorithm converge to a stationary distribution and analyze its resulting bias and variance relative to the problem's solution. We provide a first-order bias expansion in both homogeneous and heterogeneous settings. Interestingly, this bias decomposes into two distinct components: one that depends solely on stochastic gradient noise and another on client heterogeneity. Finally, we introduce a new algorithm based on the Richardson-Romberg extrapolation technique to mitigate this bias.
Fichier principal
Vignette du fichier
paper_fedavg_richardson_romberg.pdf (866.52 Ko) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04878343 , version 1 (10-01-2025)

Identifiers

  • HAL Id : hal-04878343 , version 1

Cite

Paul Mangold, Alain Durmus, Aymeric Dieuleveut, Sergey Samsonov, Eric Moulines. Refined Analysis of Federated Averaging's Bias and Federated Richardson-Romberg Extrapolation. 2025. ⟨hal-04878343⟩
0 View
0 Download

Share

More