Fault zone properties evolve during the seismic cycle and explainable machine learning can help us monitor these changes. We propose the innovative use of RGB spectrograms to represent 3-component seismic waveforms, facilitating both model training and interpretation. We apply a CNN to classify foreshocks and aftershocks from RGB spectrograms and use SHapley Additive exPlanations (SHAP) to interpret the model’s decisions. Given the CNN’s high accuracy (99.66%), we turn our attention to understanding its decision-making process through SHAP. SHAP reveals that the CNN distinguishes classes based on a narrow frequency band near 30 Hz, linked to elastic wave attenuation. The RGB format enables intuitive SHAP interpretation by preserving time-frequency structure across seismic components. Temporal SHAP analysis shows meaningful evolution in aftershocks, interpreted as fault healing, and highlights the complex and varied nature of foreshock behavior. Our work highlights SHAP as a robust and transparent method to extract physical insight from ML models and contributes to bridging data-driven approaches with seismological understanding.