Adversarial Perturbations Straight on JPEG Coefficients

Adversarial examples are samples that are close to benign samples with respect to a distance metric, but misclassified by a neural network. While adversarial perturbations of images are usually computed for RGB images, we propose perturbing straight on JPEG coefficients with the ability to individually control the perturbation applied on each color channel and frequency. We find that perturbation as a function of perceptual distance is most efficient for medium frequencies, especially when JPEG compression is used in defense. Overall, we show that attacks on JPEG coefficients are more efficient than state-of-the-art methods that (mainly) apply their perturbation in RGB pixel space. This is partly due to the use of the YCbCr color space, which allows to perturb luma information exclusively, but also due to perturbing the cosine transform coefficients instead of pixels. Moreover, adversarial training using such JPEG attacks with various frequency weighting vectors results in generally strong robustness against RGB and YCbCr attacks as well.