The Routing Protocol for Low-Power and Lossy Networks (RPL) is a commonly employed protocol in Internet of Things (IoT) settings for route management and optimization in networks with limited power and resources. However, the protocol's vulnerability to various security attacks poses substantial risks to the dependability and privacy of IoT networks. To address this issue, Deep Stacked Neuro-Fuzzy System (DSNFyS) is introduced for detecting attacks in RPL-based IoT. The process begins with RPL routing simulation, followed by attack detection at the Base Station (BS) using log data. The input log data is sourced from the designated Bot-IoT dataset. Data normalization is then carried out using the min–max normalization technique. Subsequently, the most crucial features are identified through feature selection, utilizing information gain and Support Vector Machine-Recursive Feature Elimination (SVM-RFE). The final step involves attack detection using DSNFyS, which combines Deep Stacked Autoencoder (DSA) and Adaptive Neuro-Fuzzy Inference System (ANFIS). The proposed DSNFyS achieved superior accuracy, True Positive Rate (TPR), and True Negative Rate (TNR) compared to existing protocols.