A Network Behavior Analysis Method to Detect Reverse Remote Access Trojan

Remote Access Trojan (RAT)reverse connections are secret and malicious, which are established to steal private data or be operated under hacker's command. To detect reverse RAT effectively, a network behavior-based method is introduced in this paper. We first conclude a typical network communication pattern. Then four uncorrelated network behavior features are extracted from every TCP session as the detection model input. Six supervised classification algorithms are applied on real network traffic data set to distinguish RAT and legitimate sessions. Besides detection accuracy, AUC is also used because the amount of RAT sessions is much less than normal sessions and AUC is suitable to evaluate the performance of such imbalanced problem. Detection accuracies of all test algorithms are higher than 0.92. AUC of Random Forest, SVM and Logistic Regression are higher than 0.94, which shows their ability to handle imbalanced data set. Compared to related work, the proposed method is effective on connection encrypted RAT detection, and can distinguish RAT sessions from similar normal sessions, like P2P or cloud application sessions.