CoDD: A Constraint-Based Dataset Discovery Tool for Open Data Lakes
摘要
Data lakes offer the flexibility to store large volumes of heterogeneous data with minimal curation. However, this flexibility comes at a cost: traditional keyword-based dataset discovery methods require reliable metadata such as table names or column headers, and become ineffective when this metadata is either missing or incomplete. This issue is especially pronounced in open or poorly maintained data lakes, where the quality of metadata cannot be guaranteed. In this paper, we present CoDD, a system for constraint-based dataset discovery in open data lakes. Instead of querying metadata (query-by-metadata), CoDD profiles datasets by extracting structured facts directly from the data using modular, user-definable components. Users can perform query-by-constraint searches by specifying constraints over the profiled facts in an interactive, question-driven interface. Early results from our user study show that CoDD enables users to find relevant datasets when traditional keyword-based approaches fail due to insufficient or misleading metadata. Furthermore, CoDD performs comparably well even when accurate metadata is available, demonstrating that query-by-constraint is a robust and scalable alternative for dataset discovery in open data lake environments.