Optimal Proactive Eavesdropping Scheme Based on Stackelberg Game Framework Against State-Secrecy Encoding: A Deep Reinforcement Learning Approach

This paper studies proactive eavesdropping in remote estimation systems where the eavesdropping attacker attacks sensors' ACK channels, and all sensors defend against the eavesdropping attack according to the designed state-secrecy encoding scheme and calibration scheme. Given essential analysis and proofs, a novel dynamic Stackelberg game framework and a Markov Stackelberg game framework are developed to design proactive eavesdropping schemes for the cases when the packet loss rate is entirely random or driven by a Markovian process, respectively. Utilizing state-secrecy encoding with a calibration mechanism and deep reinforcement learning, the frameworks approximate the optimal strategy of the eavesdropper based on the best response of sensors. Specifically, the objective of the optimal strategy is to minimize the minimum mean square error (MMSE) incurred when the eavesdropper decodes the transmitted message. In reinforcement learning, the action decompositions and constraints are introduced to obtain a more efficient reduction of the action space and exploration of reasonable strategies. The superiority of the proactive eavesdropping strategies derived from both game frameworks is demonstrated through numerical simulations.