CogSpy: Cognition-Driven PIN Inference Against Layout-Randomized Soft Keyboards

As mobile devices have become deeply integrated into daily life, users often input sensitive data (i.e., PINs) to unlock services or authorize payments, which introduces high risks of side-channel attacks. To defend against potential attacks, in practice, soft keyboards for PIN entry are randomized in layout to mitigate such threats. In this paper, we present CogSpy, a novel cognition-driven acoustic side-channel attack that infers PINs on randomized soft keyboards. Unlike prior work that relies on video, power, or electromagnetic emanations, CogSpy exploits a previously unexplored vulnerability: distinct human recognition latencies for symbolic numbers. By modeling cognitive latency patterns and leveraging acoustic keystroke signatures, CogSpy learns individual and digit-level recognition features through contrastive and self-supervised learning. Furthermore, we also introduce a novel Logic-Guided Inference Network that integrates recognition patterns with the reasoning capabilities of a large language model (LLM) to prune the hypothesis space and infer complete PIN sequences. We extensively evaluate CogSpy on both Android and iOS devices, and results show that it improves the probability of successful inference by up to 4000 \(\times \) , which demonstrates a practical threat to current mobile authentication systems and shows that representation learning and LLMs can enable new side-channel attacks.