Moving data analytics to the extreme edge demands for the accurate selection of the trade-off between computational complexity and power requirements. This is crucial for Structural Health Monitoring (SHM) systems where sensors are typically battery-operated and equipped with resource-constrained processors. This paper considers the possibility to perform inference by using just one sensing node, hosting a tiny Convolutional Neural Network (CNN), for the prediction of the health status of the structure from vibration data. Emphasis is given to the importance of selecting the optimal sampling parameters (length of the acquisition window and sampling frequency). Experiments on the Z24 bridge benchmark show that a tiny CNN can achieve classification scores comparable with state-of-the-art results (>96%), while avoiding data transmission. An in-depth energy profiling has been conducted after deploying the sought model on a wireless node based on an STM32L496 processor and an accelerometer. It reveals that the selection of a proper duration of the acquisition time (half than typical duration) can halve the energy consumption of the sensor per hour (from 130 mJ/h down to 31 mJ/h) while introducing a minimal drop in the prediction accuracy (less than 1%).