Proactive Cyber Defense: Multi-source Data Collection for Threat Intelligence from Dark Web, Clear Web, and Telegram
摘要
This paper explores proactive cyber defense through multi-source data collection from the dark web, clear web, and Telegram channels, focusing on the integration of diverse intelligence to enhance threat detection. It examines methodologies for collecting and processing threat intelligence, including automated crawling, scraping, and file extraction, alongside ethical and legal considerations. Additionally, the study evaluates alternative data extraction techniques, such as regex-based methods and traditional parsing, as lightweight alternatives to Large Language Models (LLMs) for handling structured and semi-structured leaked data. The findings underscore the importance of combining multi-source intelligence with efficient extraction methods to strengthen cybersecurity defenses.