The analysis and classification of diverse noises in the urban environment have emerged as a research focus. Aiming at the limitations of existing acoustic feature classification algorithms in terms of accuracy and background noise processing, this paper proposes a two-stage multi-feature acoustic scene classification network. Firstly, a secondary feature extraction module is proposed to combine the Mel-Frequency Cepstral Coefficients (MFCC) and the first-order differential MFCC for filtering the noisy signals and preliminary feature refinement. Immediately after that, in order to further improve the classification performance, the double pooling residual module is proposed, which can deeply capture the contextual feature information in the acoustic scene and enhance the network’s ability to understand the complex acoustic environment. Judging from the experimental outcomes, in contrast to the baseline network, this algorithm has boosted the classification accuracy of urban environmental sounds by 1.6%.