Deep learning has demonstrated success in domains such as vision and speech, largely due to its ability to learn hierarchical feature representations via backpropagation. However, tree-based models, such as Random Forest and XGBoost, remain dominant for tabular data, often, but not always, outperforming deep Artificial Neural Networks (ANNs) while requiring less computational resources. It has been demonstrated that neural networks are particularly effective for data that is less heavily preprocessed and that contains important yet complex feature dependencies. This paper introduces Deep Decision Forest (DDF), a multilayer decision tree ensemble that bridges this gap by incorporating a feedback mechanism analogous to backpropagation. We postulate that such a mechanism will enable tree-based models to capture hierarchical feature representations to a degree that was previously impossible. Each layer of trees produces an output vector that serves as the input features for subsequent layers, and final predictions are obtained through majority voting in the last layer. One key difference between tree-based models and ANNs is that, instead of making minor adjustments to all the numeric weights at once, DDF first identifies the most underperforming features across layers, and selectively retrains only the corresponding trees using specialised improvement datasets. Experiments on seven benchmark datasets demonstrate that DDF consistently outperforms a standard Decision Tree, performs as well as or better than Random Forest, and achieves competitive accuracy compared to Deep Forest. These experiments demonstrate that integrating elements inspired by deep learning into tree ensembles is both feasible and effective, offering a new hybrid approach for tabular learning. All the code and experiments are available open source at: https://github.com/caisr-hh/Deep-Decision-Forest .