Fingertip Air-Writing with Ambient Light

Contact-free interaction with public devices could become popular. To achieve this purpose, state-of-the-art methods mainly use cameras to capture mid-air hand gestures, which are power-hungry and can raise privacy issues. In this work, we design LightDigit, a system for fingertip air-writing of digits with ambient light and photodiodes, to enable contact-free interactions with public devices. The key enabler is detecting and interpreting dynamic shadows on photodiodes introduced by fingertip movements. We design an embedded deep learning model LightConvRNN –customized ConvRNN with attention pooling– to capture spatial and temporal patterns in the dynamic shadows. We evaluate LightDigit through extensive experiments under different light conditions. Evaluation results show that our model can achieve an accuracy of up to 98%. Through model compression, the model size is reduced by 92% with less than a 5% drop in the performance. LightDigit is robust to ambient light positions (60 \(^\circ \) ) and ambient light intensity (5000 lx).