Initial classifier weights replay for memoryless class Incremental Learning - CEA - Commissariat à l’énergie atomique et aux énergies alternatives Accéder directement au contenu
Poster De Conférence Année : 2020

Initial classifier weights replay for memoryless class Incremental Learning

Résumé

Incremental Learning (IL) is useful when artificial systems need to deal with streams of data and do not have access to all data at all times. The most challenging setting requires a constant complexity of the deep model and an incremental model update without access to a bounded memory of past data. Then, the representations of past classes are strongly affected by catastrophic forgetting. To mitigate its negative effect, an adapted fine tuning which includes knowledge distillation is usually deployed. We propose a different approach based on a vanilla fine tuning backbone. It leverages initial classifier weights which provide a strong representation of past classes because they are trained with all class data. However, the magnitude of classifiers learned in different states varies and normalization is needed for a fair handling of all classes. Normalization is performed by standardizing the initial classifier weights, which are assumed to be normally distributed. In addition, a calibration of prediction scores is done by using state level statistics to further improve classification fairness. We conduct a thorough evaluation with four public datasets in a memoryless incremental learning setting. Results show that our method outperforms existing techniques by a large margin for large-scale datasets.
Fichier principal
Vignette du fichier
Belouadah_BMVC2020.pdf (799.35 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

cea-04532679 , version 1 (04-04-2024)

Licence

Paternité

Identifiants

  • HAL Id : cea-04532679 , version 1

Citer

Eden Belouadah, Adrian Popescu, Ioannis Kanellos. Initial classifier weights replay for memoryless class Incremental Learning. BMVC 2020 - The 31st British Machine Vision Virtual Conference, Sep 2020, virtual, United Kingdom. pp.1-13, 2020, Proceedings of BMVC 2020. ⟨cea-04532679⟩
11 Consultations
4 Téléchargements

Partager

Gmail Facebook X LinkedIn More