Contextual advertising is a digital advertising method of displaying advertisements on the Internet according to the host website’s context. This leads to the need for businesses to ensure brand safety on the Internet, which is a strategy that enables a brand to avoid display of advertisement on illegal or inappropriate pages. This work proposes an effective approach to ensure brand safety online using multiclass classification to detect unsafe images for brands. We created a dataset with seven unsafe classes (adult, gore, gambling chip, gun, knife, alcohol, and cigarette) and a safe class. Moreover, some pruning techniques are investigated to observe the performance of our method. The performance is tested on certain key factors such as accuracy, model size, and inference speed. Experimental results demonstrate that our proposed approach obtains good performance and has potential to apply to real systems.