Context-Aware Neural Machine Translation Models Analysis And Evaluation Through Attention - Productions scientifiques du CLILLAC-ARP Access content directly
Journal Articles Revue TAL : traitement automatique des langues Year : 2024

Context-Aware Neural Machine Translation Models Analysis And Evaluation Through Attention

Abstract

Model explainability has recently become an active research field.Many works are published supporting or criticizing attention weights as model explanation. In this work we adhere to the former and analyze attention as explanation for Context-Aware Neural Machine Translation (CA-NMT). Since its evaluation often concerns the evaluation of models in resolving discourse phenomena ambiguity, we perform analyses and evaluations over coreference links in a parallel corpus. We propose a human evaluation over heatmaps, strengthened by a quantitative evaluation based on attention weights over coreference links and with different metrics purposely designed for this work. Such metrics provide a more explicit evaluation of the CA-NMT models than evaluations using contrastive test suites.
Fichier principal
Vignette du fichier
ReveuTAL2023_Explicabilite-2.pdf (3.21 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04581509 , version 1 (21-05-2024)

Identifiers

  • HAL Id : hal-04581509 , version 1

Cite

Marco Dinarelli, Dimitra Niaouri, Fabien Lopez, Gabriela Gonzalez-Saez, Mariam Nacklé, et al.. Context-Aware Neural Machine Translation Models Analysis And Evaluation Through Attention. Revue TAL : traitement automatique des langues, In press. ⟨hal-04581509⟩
0 View
0 Download

Share

Gmail Mastodon Facebook X LinkedIn More