The issue of adjusting neural network structure is one of the core problems in artificial intelligence. The issue of adjusting neural network structure is one of the core problems in artificial intelligence. A highly desirable scenario is a dynamic architecture that evolves structurally during the training process. In this paper, we propose a new and powerful tool that facilitates dynamic changes in network structure. We introduce a novel form of residual connections based on matrix extensions, enabling adaptable weight matrices and enhancing structural flexibility. The approach enhance the potential for structural modifications. We conducted a series of comprehensive experiments confirming that the new residual connections scheme behaves very well. The new type of connection improves performance by enabling better error flow during the error backpropagation phase, resulting in more efficient training. Our method demonstrates superior performance and enhanced trackability during the training process. The paper is supplemented by Python source code to ensure reproducibility. This method marks a significant starting point, showing immense potential for more advanced dynamic neural network models and transfer learning with dynamic models.