CS426: Big Data Processing

Final Exam: 3:40pm ~ 5:40pm, 6/14/2017 Wednesday, 东中院3-201.

Time: 12:55pm ~ 3:40pm (class 6~8), Tuesday and Friday, from week 9 to week 16
Venue: 东上院303

Teaching Assistant: Guodong Shen (email: gdshen@sjtu.edu.cn) and Jian Chen (email: cs_jerrychen@sjtu.edu.cn)

Textbook:

Anand Rajaraman and Jeffrey D. Ullman, Mining of Massive Datasets, Cambridge University Press, 2011. (book website)

Reference:

1. Holden Karau, Andy Konwinshi, Patrick Wendell, Matei Zaharia, Learning Spark, O'REILLY, 2015.

2. Nick Pentreath. Machine Learning with Spark. Packt Publishing, 2015.

3. Chuck Lam. Hadoop in Action. Manning Publications, First Edition, 2010.

4. Jiawei Han, and Micheline Kamber. Data Mining: Concepts and Techniques. Morgan Kaufmann, Second Edition, 2006.

5. Christopher M. Bishop. Pattern Recognition and Machine Learning. Springer, 2006.

Syllabus: (subject to change)

• Week 1:
  
  Course overview; slide
  A brief introduction to data-intensive scalable computing & data mining; slide
  Pthreads; slide
  OpenMP. slide



• Week 2:
  
  MPI; slide
  MapReduce; slide
  Extra-talk from TeraData;



• Week 3:
  
  Association Rules. slide
  Linear Algebra Background for SVD; slide
  Dimensionality Reduction: SVD & CUR; slide
  



• Week 4:
  
  Information Retrieval; slide
  Clustering. slide
  



• Week 5:
  
  Classification; slide
  Finding Similar Items. slide
  



• Week 6:
  
  Recommender Systems; slide
  Search Engines. slide
  



• Week 7:
  
  Dragon Boat Festival;
  Link Analysis; slide
  



• Week 8:
  
  Advertising on the Web; slide
  Mining Data Streams; slide
  Review. slide
  

Homework:

• Homework 1: OpenMP practice. Deadline: 4/27/2017.
• Homework 2: Spark practice. Deadline: 5/4/2017.
• Homework 3: Association rules & Dimensionality reduction. Deadline: 5/11/2017.
• Homework 4: Clustering for the dataset. Deadline: 5/18/2017.
• Homework 5: Classification with the training dataset and the test dataset.. Deadline: 5/25/2017.
• Homework 6: Search Engines. Deadline: 6/1/2017.
• Homework 7: Link Analysis. Deadline: 6/11/2017.

Grading policy: (subject to change)

• Homework: 40%
• Final exam: 60%