Context-aware bidirectional recurrent network with transformer encoding for text-based emotion detection

Abstract

Emotion detection is the process of recognizing the emotional state expressed through textual content. It has become an important task in Natural Language Processing (NLP) due to the growing amount of text-based communication. Moreover, the NLP techniques help to analyze textual information and classify it into different emotional categories. Traditional machine learning methods generally rely on manually selected features, while deep learning models can automatically learn features, they may not consistently capture the diverse linguistic patterns present in textual data. This may affect the accurate identification of emotions across different types of textual content. This manuscript introduces a Transformer-based Bidirectional Recurrent Network for Emotion Detection (TBiRN-ED), designed to learn complex textual representations for distinguishing different emotion categories. The proposed model employs Term Frequency-Inverse Document Frequency with Category-Distribution Weight, which integrates both term importance and category-specific information into the feature representation. These representations are further passed to the Transformer-Based Bidirectional Recurrent Network model, where transformer-based contextual encoding and bidirectional long short-term memory are combined to capture both contextual relationships and sequential dependencies among words. In addition, the Adan optimizer is utilized to improve the training process and parameter updates. The experimental analysis of the proposed TBiRN-ED framework is evaluated using the Emotions dataset and the Emotions Dataset for NLP, achieving accuracy of accuracy of 96.42 % and 95.70 % compared with existing methodologies. Additionally, an ablation study is conducted to examine the contribution of each component of TBiRN-ED to the overall classification performance. The results demonstrate its effectiveness in distinguishing different emotional categories from textual input, supporting its suitability for text-based emotion detection.