AI Skills for Engineers: Data Engineering and Data Pipelines

Week 1:

• We ask why we should care about data management for Artificial Intelligence and Machine Learning (ML) systems.
• We examine which data are needed in the ML lifecycle and what properties that data should have.
• We discuss the effort and time needed for data management activities, and look at possible data sources.

Week 2:

• The basic key concepts of data management, such as databases, data models and data schemas are all introduced.
• The Relational Data Model is explained and contrasted with the Single-Table Model (like CSV and Excel) and Document Models.

Week 3:

• We show how to extract data from existing relational databases using SQL queries and converting the query results into CSV files for further processing using pandas in Python notebooks.

Week 4:

• The different ways setoff setting up and running Python notebooks are covered, including cloud-based notebooks and local notebooks.
• We will take you step by step through the process of setting up your conda environment and installing Jupyter and pandas libraries.
• You will learn how to run notebooks in VS code.

Week 5:

• Become a pandas expert.
• Explore the essential functionalities of pandas and, most importantly, write elegant and efficient Python pandas code to process and engineer tabular data.

Week 6:

• You will learn how to make simple and clear scientific figures in Python using the Seaborn library.
• Use the core functions provided by Seaborn to make beautiful statistical plots.