Numerous fields have recently employed autonomous vehicles. In this regard, concerns about terrain classification have become critical for safe and efficient navigation. Existing studies conducted on terrain classification under different light conditions are based on proprioceptive sensors, so they are platform-dependent and cannot be implemented on other robotic platforms such as legged, hexapod, and flying robots. Furthermore, existing studies have evaluated the performance of LiDAR and camera sensors separately, or the fusion of the two sensors under different light conditions has not been specifically mentioned. The primary goal of the present work is to develop an accurate and reliable terrain classification system for different lighting conditions using a decision-level fusion of CNN and SVM classifiers’ outputs using data obtained from vision and LiDAR sensors that is independent of robot design. We use a low-resolution camera and a 1D LiDAR sensor to reduce costs in order to use it in any region, including those with limited resources. Visual and distance data were collected for grass, asphalt, sand, and concrete floors both day and night from a single premises. The weighted averaging is used as the decision-level fusion technique, and it has brought some improvement in overall classification performance. Significant gains were noticed in the daytime and night-time classifications, with an accuracy of 91.78% and 95.24%, respectively. These results provide a significant improvement over the individual CNN and SVM models that verify that the fusion of visual and LiDAR information would work effectively in obtaining a robust terrain classification.