The proliferation of abusive websites, particularly those facilitating phishing, fraud has emerged as a critical cybersecurity threat. Detecting these abusive websites efficiently remains a crucial challenge, necessitating sophisticated feature engineering and advanced machine learning techniques. In this paper, we present a comprehensive comparative study of domain-based and content-based approaches for abusive website detection with two datasets such as Vietnamese abusive websites and international phising datasets. Through extensive evaluation, we demonstrate that the integration of multiple feature types significantly enhances the detection accuracy. In particular, hosting-related features exhibit strong independent predictive capability, while machine learning models that take advantage of these features continue to achieve robust performance. Although extracted features contribute substantially to high-accuracy detection, our findings indicate that source code analysis is the most effective method for identifying abusive websites. In particular, language models, such as Phishlang, excel at capturing the textual patterns within website source code, achieving outstanding performance with an accuracy of 0.98 and an F1-score of 0.97.