The integration of neural networks and decision trees has emerged as a promising direction in machine learning, addressing the limitations of each approach while leveraging their respective strengths. A novel hybrid architecture is proposed that integrates specific decision trees, optimizing arbitrary differentiable loss functions, with neural networks. The neural network serves a dual role: generating differentiable loss functions for training the decision tree and enhancing the model’s predictive accuracy. The neural network and the tree are trained by using a gradient-based algorithm in an end-to-end manner. The hybrid architecture supports flexible loss functions, making it applicable to diverse tasks. Numerical results demonstrate that the model outperforms gradient boosting with several models handling tabular data, achieving state-of-the-art performance in classification and sequence processing tasks. The code implementing the proposed architecture is publicly available.