Machine Learning: Foundations (2012/13)

Tentative Class schedule:

1. Introduction
2. Bayesian Inference [slides]
3. PAC model and Occam Razor
4. Online Learning: Mistake Bound, Winnow, Perceptron.
5. Regret Minimization
6. Boosting and Margin
7. Nearest Neighbor
8. VC dimension I - definition and impossibility result [ppt]
9. VC dimension II - sample bound (Rademacher complexity)
10. Convex Programming and Support Vector Machine [Andrew Ng class notes]
11. Kernels, SVM and SMO algorithm
12. Regression Models
13. Model Selection

Homework

submission guidelines.

Homework 1

Homework 2

Homework 3

Homework 4

Data Sets

Iris

mnist

ISOLET

Glass

BOOK

Foundations of machine learning Mehryar Mohri, Afshin Rostamizadeh and Ameet Talwalkar; MIT Press, 2012

SCRIBE

Scribe notes: Each student will write a scribe note for a lecture (template [pdf,tex] explanation on Latex [pdf,tex])

Scribe list

FINAL PROJECT

STUDENT PROJECTS

Courses on Machine Learning Elsewhere:

• Introduction to machine leaning - Shai Shalev-Shwartz (HUJI)
• Machine Learning Theory – Maria Florina Balcan (Georgia Tech)
• Machine Learning Theory – Avrim Blum (CMU)
• Statistical Learning Theory – Peter Bartlett (UC Berkely)
• Machine Learning – Andrew Ng (Stanford)
• Machine Learning – Tommi Jaakkola and Michael Collins (MIT)
• Foundations of Machine Learning - Mahryar Mohri (NYU)

PPTX Presentations (students)

Nearest Neighbor

Bayes Inference

SVM

VC dimension

AdaBoost

PAC