Machine Learning: Foundations (2010/11)

Tentative Class schedule:

1.       Introduction

2.       Bayesian Inference

3.       PAC model and Occam Razor

4.       Online Learning: Mistake Bound, Winnow, Perceptron. [pptx]

5.       Regret Minimization

6.       Boosting and Margin

7.       VC dimension I - definition and impossibility result [ppt]

8.       VC dimension II - sample bound (Rademacher complexity)

9.       Convex Programming and Support Vector Machine [Andrew Ng class notes] [pptx]

10.   Kernels, SVM and SMO algorithm

11.   Model Selection

12.   Decision Trees

13.   Fourier transform of Boolean functions [survey]


submission guidelines.

Homework 1 [note that the programming can be done also in Matlab] comments

Homework 2

Homework 3

Homework 4


Data Sets






Scribe notes: Each student will write a scribe note for a lecture (template [pdf,tex] explanation on Latex [pdf,tex])

Scribe list


Courses on Machine Learning Elsewhere:

         Introduction to machine leaning - Shai Shalev-Shwartz (HUJI)

         Machine Learning Theory Maria Florina Balcan (Georgia Tech)

         Machine Learning Theory Avrim Blum (CMU)

         Statistical Learning Theory Peter Bartlett (UC Berkely)

         Machine Learning Andrew Ng (Stanford)

         Machine Learning Tommi Jaakkola and Michael Collins (MIT)

         Foundations of Machine Learning - Mahryar Mohri (NYU)