Hear to reveal: Stealing keystroke content from keyboard acoustic side-channel

With the widespread application of high-precision microphones in various smart terminals, attackers can eavesdrop on users’ keystroke content. Although existing attack strategies have made some progress in specific scenarios, they generally face practical deployment bottlenecks such as insufficient generalization ability and strong dependence on environment and data. To reveal the real threat of attacks based on sound signals, we propose a method for stealing keystroke content from the acoustic side-channel on keyboards, named HERO. First, we improve the ability to detect keystroke fragments from complex backgrounds through a multi-scale energy fusion strategy. Then, through self-supervised pre-training using spectral masking and feature reconstruction, HERO learns key representations from keystroke audio. Finally, we guide HERO to learn the distinguishing keystroke features based on discrimination tasks. In particular, labeled keystroke samples are used to improve HERO's classification performance in the fine-tuning stage. Extensive experiments across mainstream keyboards demonstrate that HERO outperforms existing baselines on multiple evaluation metrics. Its robustness is further verified by varying noise levels, recording distance, recording position, and microphone type. In real-world scenarios, HERO achieves 97.5% accuracy in word input tasks and 93% accuracy in numeric password input tasks. Under a strict leave-one-out protocol with 18 participants, HERO achieves 91.48% average accuracy, demonstrating its promising cross-user generalization ability. These results validate HERO's adaptability and attack effectiveness, highlighting the serious threat that keyboard acoustic side-channel attacks pose to user privacy.