Designing optimal neural network architectures for object classification can be time consuming. This paper presents an Evolutionary Neural Architecture Search (ENAS) approach using mutation-based generation to automate model design for wearable sensor data classification. Unlike traditional hyperparameter tuning methods such as grid search, which rely on exhaustive combinations of predefined models, the proposed ENAS approach evolves lightweight and accurate models by iteratively mutating high-performing architectures. Using data from IMUs on the thumb and index finger during object manipulation, our method achieved 95% accuracy with 80.34 ms inference time on an embedded device. These results highlight ENAS as an effective solution for real-time, computationally constrained wearable applications.