Overview of the HASOC Subtrack at FIRE 2021: Hate Speech and Offensive Content Identification in English and Indo-Aryan Languages

Mandl, T and Modha, S and Shahi, GK and Madhu, H and Satapara, S and Ranasinghe, T and Zampieri, M (2021) Overview of the HASOC Subtrack at FIRE 2021: Hate Speech and Offensive Content Identification in English and Indo-Aryan Languages. In: 13th Annual Meeting of the Forum for Information Retrieval Evaluation, FIRE 2021, 13 - 17 December 2021, Virtual, Online, pp. 1-19.

PDF FIRE-WN_2021.pdf - Published Version Restricted to Registered users only Download (193kB) | Request a copy

Abstract

The widespread of offensive content online such as hate speech poses a growing societal problem. AI tools are necessary for supporting the moderation process at online platforms. For the evaluation of these identification tools, continuous experimentation with data sets in different languages are necessary. The HASOC track (Hate Speech and Offensive Content Identification) is dedicated to develop benchmark data for this purpose. This paper presents the HASOC subtrack for English, Hindi, and Marathi. The data set was assembled from Twitter. This subtrack has two sub-tasks. Task A is a binary classification problem (Hate and Not Offensive) offered for all three languages. Task B is a fine-grained classification problem for three classes (HATE) Hate speech, OFFENSIVE and PROFANITY offered for English and Hindi. Overall, 652 runs were submitted by 65 teams. The performance of the best classification algorithms for task A are F1 measures 0.91, 0.78 and 0.83 for Marathi, Hindi and English, respectively. This overview presents the tasks and the data development as well as the detailed results. The systems submitted to the competition applied a variety of technologies. The best performing algorithms were mainly variants of transformer architectures.

Item Type:	Conference Paper
Publication:	CEUR Workshop Proceedings
Publisher:	CEUR-WS
Additional Information:	The copyright for this article belongs to CEUR-WS.
Keywords:	Classification (of information); Deep learning; Fires; Social networking (online); Text processing, Content identifications; Deep learning; Evaluation; Hate speech; Machine-learning; Multilingual text classification; Multilingual texts; Offensive languages; Social media; Text classification, Speech
Department/Centre:	Division of Electrical Sciences > Electronic Systems Engineering (Formerly Centre for Electronic Design & Technology)
Date Deposited:	03 Aug 2022 12:03
Last Modified:	03 Aug 2022 12:12
URI:	https://eprints.iisc.ac.in/id/eprint/75283

Actions (login required)

View Item