Share
Export Citation
Modified Encoder Transformer Architecture for Hate Speech and Abusive Language Detection in Indonesian-English Code-Mixed Text
Mutmainnah R.
Proceedings of the 23rd IEEE International Conference on Computer Applications Icca 2026
Abstract
Social media makes it easy to share information, but it also allows unfiltered content to spread quickly, leading to misunderstandings. So, we need to automate systems that can detect hate speech and abusive language, especially in codemixed text. Switching between languages within a single sentence often introduces ambiguity, making it difficult for standard Transformer models to maintain context consistently. This research contributes to modifying the Transformer encoder architecture and presents a mechanism that regulates the flow of information between layers. This mechanism helps the model select and suppress truly relevant information at each processing stage, resulting in a more accurate understanding of code-mixed text. Test results show that the proposed approach achieves 89.32 % accuracy and outperforms both BERT and the baseline Transformer encoder. These findings demonstrate that the modified architecture is more effective in handling the linguistic complexity of multilingual texts and can support the development of more reliable toxic content detection systems in modern social media environments.