In the not-so-distant past, most data teams fully relied on data transformation tools that were built around drag-and-drop graphical user interfaces (GUIs) or stored procedures. Unfortunately, implementations of these solutions tend to end up being very complex and in a consistently unreliable state. One of the main reasons for this is that software engineering best practices, which are also applicable to data teams, are not used. For example, data transformations tend to not be modular, data quality testing is infrequently implemented, and version control is not used.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Models

  • Dustin Dorsey,
  • Cameron Cyr

摘要

In the not-so-distant past, most data teams fully relied on data transformation tools that were built around drag-and-drop graphical user interfaces (GUIs) or stored procedures. Unfortunately, implementations of these solutions tend to end up being very complex and in a consistently unreliable state. One of the main reasons for this is that software engineering best practices, which are also applicable to data teams, are not used. For example, data transformations tend to not be modular, data quality testing is infrequently implemented, and version control is not used.