Seq-to-NSeq model for multi-summary generation

Guillaume Le Berre (1) , Christophe Cerisara (1)

1 SYNALP - Natural Language Processing : representations, inference and semantics

Summaries of texts and documents written by people present a high variability, depending on the information they want to focus on and their writing style. Despite recent progress in generative models and controllable text generation, automatic summarization systems are still relatively limited in their capacity to both generate various types of summaries and capture this variability from a corpus. We propose to address this challenge with a multi-decoder model for abstractive sentence summa-rization that generates several summaries from a single input text. This model is an extension of a sequence-to-sequence model in which multiple concurrent decoders with shared attention and embeddings are trained to generate different summaries that capture the variability of styles present in the corpus. The full model is trained jointly with an Expectation-Maximization algorithm. A first qualitative analysis of the resulting de-coders reveals clusters that tend to be consistent with respect to a given style, e.g., passive vs. active voice. The code and experimental setup are released as open source.

Domaines: Traitement du texte et du document

Guillaume Le Berre, Christophe Cerisara. Seq-to-NSeq model for multi-summary generation. ESANN 2020, Oct 2020, Bruges, Belgium. ⟨hal-02902734⟩