Finding Data on the Web, Data Sets, Web Scraping, Web API

This chapter discusses important aspects of collecting data from the web for Data Science projects and data driven projects, including such sources as dataset collections, data dumps, Web API, and the web itself. Familiarity with this topic and the ability to effectively collect data from the web, also called webscraping, is a demanded competence and skill in Data Science and Big Data job vacancies. Collecting data from external sources and from the web is a necessary component of educational projects on Big Data Infrastructure technologies that allow the students to prepare for practical work in their postgraduate careers. The chapter also provides best practices in WebAPI design.