Big Data for Agri-Food: Principles and Tools

Wageningen University via edX

edX based on 9 ratings

Likes

Overview

Demystify complex big data technologies Compared to traditional data processing, modern tools can be complex to grasp. Before we can use these tools effectively, we need to know how to handle big data sets. You will understand how and why certain principles such as immutability and pure functions enable parallel data processing (divide and conquer), which is necessary to manage big data. During this course you will acquire this principal foundation from which to move forward. Namely, how to recognise...

Syllabus

Module 1: Big data definition and characteristics In module 1, you will learn how to recognize the characteristics of a big data problem in agriculture, to see where its biggest challenge lies. Should the solution focus on size, speed, various formats or uncertainty of data? Should you scale up or scale out?
Module 2: Big data principles: what are they and why do we need them In module 2, you'll learn the principles that are required for scaling out: immutability and pure functions, and map-reduce. What are these and why do we need them?
Module 3: Bring those principles to practice Module 3 shows you how to bring those principles into practice. You will learn what a cluster is, and how a distributed file system in a client-server architecture works, with Hadoop. You will understand why such a system is indeed scalable.
Module 4: Big data technologies that make implementation so much easier Module 4 goes further into the application of big data technology, the big data stack of technologies". The main message here is that if you know what you want to do, these technologies can take the work out of your hands. For example, you will see Apache Spark, a big data technology platform, that applies map-reduce for you.
Module 5: The big data workflow and pipeline; the how and why of datalakes Module 5 dives deeper into the data. You'll learn about datalakes and why a datalake is different from a traditional database. You'll understand what a big data workflow looks like and what a pipeline is.