De²Trojan: Deployable Trojan Analysis Tool and Benchmark for the Machine Learning Lifecycle via Decoupling

Trojans (backdoors) are known to raise critical security concerns for deep neural networks in machine learning (ML) systems. Despite the extensive backdoor methods and benchmarks, existing research overlooks the perspective of the ML lifecycle (i.e., the entire process from system design to data collection to model deployment). To address this gap, this paper introduces DE²TROJAN, a Deployable Trojan Analysis Tool via Decoupling, which establishes a standardized pipeline to investigate backdoor attacks and defenses within the ML lifecycle. DE²TROJAN decouples the attack surface from the general ML process through a stage-first hijacking approach, using an abstract interface for ML lifecycle stages to enhance the deployability to the ML lifecycle. Besides, its benefits are two-fold: 1) facilitating the systematic analyses of multi-stage attacks/defenses and their combinations, shedding light on how to improve attack and defense strategies. For example, we find that current attacks (defenses) are not effective in continuous scenarios, and combining attacks (defenses) at different stages improves their effectiveness from 30.11% (8.63%), the worst cases, to 90.27% (68.73%) and 2) making it possible to identify potentially vulnerable stages, especially when iteratively updating the model in ML lifecycle. For example, we identify that backdoor attacks in the data collection stage are more vulnerable than expected, and it is more difficult to remove them from the ML lifecycle. To eliminate the impact of such attacks, it is most effective to apply backdoor defense during the deployment stage, in addition to cleaning the data before training. Overall, we present a comprehensive benchmark of backdoors within the ML lifecycle, involving 20 representative attacks and defenses, as well as their combinations, using 11 evaluation metrics.