SmartInsight: Learning-Based Automatic Insight Discovery for Exploratory Data Analysis

By pruning the search space of insight discovery based on some heuristic rules, automatic insight discovery can alleviate the heavy burden and spare unnecessary exploratory trials in exploratory data analysis (EDA). However, these rule-based methods cannot guarantee the quality of the discovered insights since the rules cannot cover all possible relationships between the data distribution and insights. To address this problem, this paper proposes a learning-based automatic insight discovery method SmartInsight. In SmartInsight, we first use a subspace-aware encoding approach to encode the data scopes that may contain insights, and train a machine learning model to capture the relationship between these data scopes and insights. Then, when discovering insights, SmartInsight predicts the scores of all possible data scopes and leverages them to prune the insight search space. Experimental results on both real-world datasets and user studies show that SmartInsight outperforms the existing automatic insight discovery methods.