Exponential growth in sports videos brings a potential benefit to the advertisers, who can advertise their content to the maximum audience during live broadcasts. However, viewers are usually more interested in sports videos rather than advertisements. Manual editing of sports broadcasts for advertisement removal is a taxing job, therefore, this paper presents an audio-visual features-based approach for advertisement detection. The proposed research exploits two facts: (1) advertisement shots contain smaller frames-count than game shots, and (2) advertisement shots comprise both music and speech content. Framework comprises two phases. In the first phase, candidate shots are short-listed based on the frame-count threshold. The second phase considered audio signal of residual shots to filter the audio frames containing the music content (marked as advertisement shots). For this purpose, a fusion of spectral features is employed to train the K-nearest neighbor for advertisement detection. Experimental results show the significance of the proposed method for advertisement detection.