A comparative study of different state-of-the-art NLP models for efficient automatic hate speech detection
Résumé
Hate speech (HS) is legally punished in many countries. Manual moderation of hate messages on social networks is no longer possible due to the huge number of messages posted every day. Automatic methods are needed to remove harmful messages. In this article, we are interested on HS detection (HSD) in social media (Twitter). Hateful content is more than just keyword detection. Hate may be implied, tweets can be grammatically incorrect and the abbreviations and slangs may be numerous. In this condition, the HSD is a complex task.Recently, natural language processing (NLP) methods have been proposed for the detection of HS. In particular, systems based on deep neural networks (DNN) have been shown to have notable performance for HSD.In this paper, we study different state-of-the-art NLP models for the HSD task. Indeed, powerful new models based on transformers have recently emerged in the literature, such as BERT, HateBERT, BERTweet, SemBERT and USE (Universal Sentence Encoder). These models are trained on different generic corpora collected from various sources. These BERT-based models can be fine-tuned for a specific task. Some models have particularities. The BERT, HateBERT and BERTweet models can be used to extract features at words or sentence level. The SemBERT model models only word-level features and further incorporate explicit contextual semantics. The USE model generates sentence-level features. The HateBERT model is trained on a Reddit corpus with a high potential of hateful content. The BERTweet model is trained on more than 840M tweets. The goal of this article is to study the generalizability of these models on HSD in tweets and to investigate the impact of sentence-level and word-level features.We investigate different DNN structures for HSD using the transformer-based models. The impact of word and sentence-based methods was assessed. Our experiments were performed on two HS corpora extracted from Twitter: Founta and Davidson (Founta et al, 2018; Davidson et al, 2017). The best performances were obtained with the BERTweet model for the two corpora (77.3% and 78.5% macro-F1 on the Founta and Davidson test sets respectively). This work was performed in the context of the French-German ANR-DFG project M-PHASIS.