PyExplore 2.0: Explainable, Approximate and Combined Clustering Based SQL Query Recommendations

While the benefit of data exploration becomes increasingly more prominent, factors such as the data volume and complexity and user unfamiliarity with the database contents, make querying data a non-trivial, time-consuming process. The big challenge for users is to find which query to ask at any point. PyExplore is a data exploration framework that aims to help users explore datasets by providing SQL query recommendations. The user provides an initial SQL query and then pyExplore provides new SQL queries with augmented WHERE clause. In this paper, we extend pyExplore with four new workflows one for approximate query recommendations, one for explainable query completions, one for combined explainable and approximate recommendation and finally a sampled decision tree workflow that is similar to pyExplore’s original workflow but this time only a small portion of the dataset gets processed. The purpose of the explainable workflows is to provide recommendations that are intuitive to the end user while the purpose of approximate workflows is to significantly reduce execution time compared to the full workflow. We evaluated the four workflows in terms of execution time and speedup compared to the full workflow. We found out that a) the quality of the approximate recommendations is on-par with the full workflow b) the explainable workflow is faster than using a decision tree classifier to produce the queries c) the approximate workflow is significantly faster than the full workflow.