Different sources of variation affect real-world acoustic scene (environmental sounds) data, including different recording locations and devices. This work proposes a fusion-based framework that uses local acoustic scene information from convolutional neural network (CNN)-based features and global acoustic scene information from transformer-based features to construct fused acoustic scene representations that contain the best of both worlds. Additionally, this work uses orthogonal projection loss for acoustic scene classification and demonstrates its effectiveness in reducing variations due to differing recording conditions (devices, locations). We evaluate the performance of the proposed fusion-based and reduced variation acoustic scene representations for the detection and classification of acoustic scenes and events (DCASE) 2020 dataset. The fusion framework enhances the scene classification performance by 7% compared to the non-fusion method, with an additional 2% improvement achieved by incorporating orthogonal projection loss. This incorporation also reduces variations due to different recording conditions, as confirmed by evaluating the proposed system on two auxiliary tasks: device classification and location classification.