Course Overview

The course covers theory and practice of pattern recognition, emphasizing concepts and practices that are common to a broad range of applications and technologies. The course is fundamental for anyone whose work involves statistical signal processing, recognition, or fault detection. The course material provides a core for work in speech and image applications with advanced recognition technologies, and finds application in anomaly detection, decision theory, timeseries analysis, and statistical signal processing.

Pattern recognition technology finds its way into a multitude of commercial applications including: medical tests, credit card fraud detection, data mining, manufacturing test, speech recognition systems, handwriting and optical character recognition, fault detection, and financial analysis to name a few. If you're planning to work, or already work in statistical signal processing or recognition, or if you simply want to know how pattern recognition technology works, this course will provide a valuable addition to your bag of tools. The statistics you will learn have even broader application.

Topics include: random vectors, detection, ROC curves, parametric and non-parametric density estimation and classification models, empirical error bounds, Bayesian estimation, and feature extraction.

Students will develop familiarity with statistical tools for developing and assessing pattern recognition systems, and apply principles to real-world examples.

Prerequisites:  The course assumes familiarity with college linear algebra and calculus, and at least one undergraduate probability and statistics course.

   Viewing Linked Material

   MATLAB Resources

MATLAB home page

MATLAB Tutorials:

1. Quick, local tutorial
2. University of New Hampshire
3. Carnegie Mellon University

• Pima Indians diabetes data -- description, data
• Wisconsin breast cancer data -- description, data
• Iris data -- description, irisDev.dat, irisTest.dat
• New Pima data for neural network exercise --- newPimaDevB.dat, newPimaTestB.dat
• Data for bootstrap exercise --- dataForBootstrap.txt
  

   Other Course Materials

• Course Overview pdf file.
• Course Syllabus pdf file
  
• Matlab Example -- Generating samples from a normal distribution with specified mean vector and covariance matrix. pdf file,
• Mathematical Preliminaries
• Random variables and linear transformations. pdf file.
• Constrained optimization using Lagrange multipliers pdf file.
• Netlab --- Matlab neural network tools  introduction
  

    Assignments

• HW 1, due Thursday, April 24
• Solution: Problem 1, Problem 2
  
• HW 2, due Tuesday, April 29
• Solution
  
• HW3, due Thursday, May 8
• Solution
  
• HW4-Midterm, due 12 NOON, SATURDAY, MAY 17
• Solution problem 1
• Solution problems 2-4
  
• HW5, due Thursday, May 29, 2008
• Matlab code for HW5
• Code to fit Gaussian misture models by EM mixturefit2.m
• Function pxi.m (called by mixturefit2)
• Example script to plot results of mixture model fitting plotresults.m
• Data for HW5 toydata1.txt
• SOLUTION p1-2, p3
• HW6-Final, due Friday, June 13, 2008
  

• Lecture 1 Overview
• Lecture 1B Math Review
  
• Lecture 2  Bayes Classifiers.  For lectures 2-4, READ Duda, Hart, and Stork 2.1-2.9
  
• Lecture 2 Supplement
• Lectures 3-4 Bayes Classifiers, continued.
• Lecture 5 Multihypothesis tests, reject option, sequential test
  
• Lecture 5b Application of sequential likelihood ratio test
• Lecture 6 Error Bounds, Single Hypothesis Tests
  
• Lecture 7 Parametric Classifiers
• Lecture 8 Criterion Functions and Bayes Posteriors
• Lecture 9 Model Estimation, Bias/Variance Decomposition
• Lecture 10 Estimation --- Maximum Likelihood and MAP
  
• Lecture 11 Map and Bayesian Estimation
• Lecture 12 EM Algorithm & Mixture Density Models
  
• Lecture 13 Kernel Density Estimation
  
• Lecture 14 k-Nearest Neighbor
  
• Lecture 15 Clustering
  

LECTURES BELOW ARE TODD'S NOTES, NOT NECESSARILY IDENTICAL TO ZAK'S

• Lecture 11  Map and Bayesian Estimation
• Lecture 12  Mixture Density Models and EM
• Lecture 13  Kernel Density Estimates
• Lecture 14  K-Nearest neighbor density estimates, Orthogonal Function Expansions
• Lecture 15  Non-parametric classifiers
• Lecture 16  Clustering and Mixture Models
• Lecture 17  Principal Component Analysis - REVISED
  
• Lecture 18  Sampling, Cross-Validation, Bootstrap
• Lecture 19 & 20 Neural Networks 1, NN 2