Assessing whether durian fruit is mature enough for harvesting is an important task. Currently, this task is often done manually by experienced experts. This study proposed a classification system of durian maturity before harvesting based on acoustic characteristics and machine learning for objective and experience-free assessment. A conventional microphone was placed into a sound-insulated tube to minimize the impact of surrounding noise at the durian orchard. A single-board computer was used to perform sound recording, analysis, and classification of durian maturity based on several popular machine learning models. Among the tested models, the K-Nearest Neighbors model revealed the best performance with accuracy and precision of 95.4% and 92.7%, respectively. This result shows that the proposed system has great potential for objectively classifying the maturity of durian fruit before harvesting.