Binary Focal Loss Enhanced LSTM for Colloquial Online Toxicity Detection

Abstract

It is difficult to detect toxicity in colloquial text, as most discussion texts found online include slang, spelling variants, non-standard grammar, implicit toxicity and emotionally charged expressions. Such traits hinder the ability of an automatic moderator to effectively handle these categories, particularly if they are not evenly distributed throughout the dataset. The present study explores various deep learning models to detect online toxicity in colloquial speech across the Conversation AI dataset of Wikipedia talk-page comments (159,571) that includes several indicators of toxicity. A text preprocessing pipeline – text cleaning, text normalization, tokenization, sequence padding, embedding representation – is done before model training. A shallow feed-forward neural network, recurrent neural network configurations and various Long Short-Term Memory models are compared under a uniform evaluation setting. To overcome the issue of class imbalance and to preserve the sequential information, an LSTM based model with a dense layer is trained using binary focal loss. The accuracy, precision, recall, F1-score, loss, training time and testing time of the evaluated models are compared. In terms of the observed performances, the proposed LSTM_50_Dense_25_Binary_Focal_Loss model has the best performance with test accuracy of 99.996% and test loss value of 0.013, training time of 90 minutes and testing time of 10 minutes among the tested architectures. The result is analysed in conjunction with the per-label performance, observations in the confusion matrix, ablation-based comparison and benchmarking with recent transformer-based models to give a more balanced assessment of classification reliability. The results indicate that the combination of LSTM-based sequential modelling and binary focal loss can help increase the accuracy of detecting toxicity in informal and imbalanced online text classification tasks.