Spark for Machine Learning & AI

Introduction

• Welcome

1. Introduction to Spark and MLlib

• Introduction to Spark
• Steps in the machine learning process
• Install Spark
• Organizing data in DataFrames
• Components of Spark MLlib

2. Data Preparation and Transformation

• Introduction to preprocessing
• Normalize numeric data
• Standardize numeric data
• Bucketize numeric data
• Tokenize text data
• TF-IDF
• Summary of preprocessing

3. Clustering

• Introduction to clustering
• K-means clustering
• Hierarchical clustering
• Summary of clustering techniques

4. Classification

• Introduction to classification
• Preprocessing the Iris data set
• Naive Bayes classification
• Multilayer perceptron classification
• Decision trees classification
• Summary of classification algorithms

5. Regression

• Introduction to regresssion
• Preprocessing regression data
• Linear regression
• Decision tree regression
• Gradient-boosted tree regression
• Summary of regression algorithms

6. Recommendations

• Understand recommendation systems
• Collaborative filtering

Conclusion

• Tips for using Spark MLlib