Math 285 Course Page
Back to my homepage

MATH 285: Classification with Handwritten Digits

Instructor: Guangliang Chen Spring 2016, San Jose State University image

Course description [syllabus] [preview]

image This is an advanced topics course in the machine learning field of classification, with the goals of introducing

  1. Dimensionality Reduction
  2. Instance-based Methods
  3. Discriminant Analysis
  4. Logistic Regression
  5. Support Vector Machine
  6. Kernel Methods
  7. Ensemble Methods
  8. Neural Networks

all based on the benchmark dataset of MNIST Handwritten Digits.

The course was motivated by a Kaggle competition - Digit Recognizer - and the Fall 2015 CAMCOS at SJSU, and thus has a data science competition flavor. Students will be asked to thoroughly test and compare the different kinds of classifiers on the MNIST dataset and strive to obtain the best possible results for those methods. The use of the MNIST handwritten digits for teaching classification was partly inspired by Michael Nielsen's free online book - Neural Networks and Deep Learning, which notes explicitly that this dataset hits a ``sweet spot'' - it is challenging, but ``not so difficult as to require an extremely complicated solution, or tremendous computational power''. In addition, the digit recognition problem is very easy to understand, yet practically important. To understand more about the rational behind this course, please refer to a reflective linkedin post by the instructor.

Course progress

DateSlidesFurther Reading
1/28 Introduction  
2/2 Lec 1: Instance-based classifiers Prof. Veksler's lecture on kNN classification
2/9 Lec 2: Principal Component Analysis (1) Instructor's notes (2) A tutorial paper
2/18 Lec 3: Discriminant analysis (for dimensionality reduction) Prof. Veksler's lecture
2/25 Lec 4: Discriminant analysis (for classification) Prof. Li's lecture
3/8 Lec 5: Two-dimensional LDA (1) Original paper (2) A note on 2DLDA
3/10 Lec 6: Logistic regression (1) Prof. Howbert's presentation (2) Andrew Ng's lecture notes
3/22 Lec 7: Support vector machines (1) Prof. Howbert's presentation (2) Prof. Veksler's lecture (3) A tutorial paper by C. Burges (4) Lagrange duality
4/21 Lec 8: Classification trees and ensemble learning (1) Trevor Hastie's talk (2) Adele Cutler's lecture
4/26 Lec 9: Neural networks (1) Prof. Veksler's lecture (2) Free online book by Michael Nielson
5/10 Poster session (introduction) See Midterm Projects section below
5/12 Oral presentations See Final Projects section below

Midterm projects (topic summary)

  1. Instanced-based classifiers (by Yu Jung Yeh and Yi Xiao)
  2. Discriminant analysis classifiers (by Shiou-Shiou Deng and Guangjie He)
  3. Two dimensional LDA (by Xixi Lu and Terry Situ)
  4. Logistic regression (by Huong Huynh and Maria Nazari)
  5. Support vector machine (by Ryan Shiroma and Andrew Zastovnik)
  6. Ensemble methods (by Mansi Modi and Weiqian Hou)
  7. Neural networks (by Yijun Zhou)

Final projects [instructions]

Some ideas include kernel knn, kernel discriminant analysis, application to USPS handwritten digits, tuning the penalty parameter C in SVM, and deskewing of MNIST digits. Talk to me if you need more ideas or have specific questions.

More learning resources

Handling singularity in LDA/QDA


Programming languages


Useful course websites


Data sets


Instructor feedback

This is an experimental course in data science, being taught at SJSU for the first time. Your feedback (as early as possible) is encouraged and greatly appreciated, and will be seriously considered by the instructor for improving the course experience for both you and your classmates. Please submit your annonymous feedback through this page.
Visitor count (since May 12, 2016): Hit Web Stats
Fast Counters