Natural language processing for extracting consumer sentiment dynamics through multimodal social media analysis to predict microeconomic consumption pattern shifts
摘要
Traditional economic predictions rely on lagging indicators that miss the early signals of change in consumption patterns, effectively robbing businesses and policymakers of strategic vision. This study seeks to fill this significant gap by developing SENTIMENT-ECON, a microeconomic modeling framework for prediction that applies natural language processing and multimodal analytics to extract consumer sentiment dynamics from social media sites. We use architectures based on transformers from deep learning in time-series forecasting. We establish causal relationships between digital expression patterns and subsequent actual market behavior. Our SENTIMENT-ECON neural architecture allows us to apply over five years (2018–2023) of analysis of 17.3 million multi-platform posts to demonstrate an unprecedented predictive accuracy of RMSE = 0.031 and MAPE = 2.7% on consumption pattern shifts across market segments. The framework predicted changes in consumption patterns and 3.8 weeks ahead of conventional indicators, and was particularly successful in discretionary categories (94.2% success rate). We use a bidirectional attention mechanism that identifies how text, image and video sentiment overlap and reinforce one another. A temporal convolutional network (TCN) allows for improved detection of new consumption trends before they arise within traditional datasets. When we compare our method to the well-known sentiment indices, we find that it is 37% more accurate at predicting the future and also 42% faster. According to this research, it may well be the case that sentiment on social media represents one of the best indicators of the economy, and it should be incorporated into BIs or SCs.