# Efficient Identification of Malicious Traffic in TLS Networks Using Machine Learning > Muttaqien H. URL kanonis: https://discover.unhas.ac.id/publications/efficient-identification-of-malicious-traffic-in-tls-networks-using-machine-lear Jurnal / Konferensi: 2025 IEEE International Conference on Artificial Intelligence and Mechatronics Systems Aims 2025 Tahun terbit: 2025 DOI: https://doi.org/10.1109/AIMS66189.2025.11229622 Citations: 0 ## Authors - Muttaqien H. ## Abstract The increasing implementation of Transport Layer Security (TLS) for encrypted communication, although improving data security and privacy, has posed challenges for network security monitoring. Conventional Network Intrusion Detection Systems (NIDS) exhibit diminished efficacy when cybercriminals employ TLS to obscure illicit activities. This research uses Suricata logs processed based on Flow ID and TLS packets labeled using Suricata rules, resulting in a dataset of 30 million records. This dataset is then used to present a machine learning (ML) method for identifying malicious TLS traffic. To train a hybrid Random Forest (RF) and Long Short-Term Memory (LSTM) model, we extract significant characteristics such as JA3 fingerprints, TLS handshake abnormalities, and statistical flow metrics. The RF-LSTM model surpasses individual machine learning models without incurring computational overhead, with a success rate of $99.9 \%$. The proposed method provides a real-time, scalable, and efficient approach to analyzing TLS traffic in cybersecurity applications. ## Keywords - Computer science - Handshake - Transport Layer Security - Random forest - Machine learning - Artificial intelligence - Identification (biology) - Intrusion detection system - Network packet - Encryption - Support vector machine - Artificial neural network - Network security - Data mining - Layer (electronics) - Deep learning - Intrusion - Ensemble learning - Feature extraction - Computer security - Deep packet inspection - Convolutional neural network - Feature (linguistics) - Data security - Malware - Computer network - Training set - Data modeling --- Sumber: Discover Unhas — RIMS Universitas Hasanuddin. Saat mengutip, gunakan DOI bila tersedia atau URL kanonis di atas.