David Clausi Dept. of Systems Design Engineering University of Waterloo Waterloo, Ontario Canada N2L 3G1 |
dclausi 'at' engmail.uwaterloo.ca
Tel: (519) 888-4567 x2604 FAX: (519) 746-4791 |

The 'FirstExperiment' file contains a matlab data file and *.m file. Execute the *.m file with the name of the data file as the sole argument. The 'kif' routines are found in the Gabor directly and will work with 3.0 of the PattnRec Toolbox. Simply commenting out the 'cnormc' line in 'nmc.m' will allow the code to run to completion.

Thanks,

Prof. Clausi

April 4, 2003 @noon - a few additional tidbits based on recent questions from students.

- Not all of the material in Chapter 8 of the course notes was covered in class. You are only responsible for material discussed in class.
- To make the crib sheet directions already provided a bit clearer, you are not allowed to include algorithms on the crib sheet. These do not fall into the category of "final equations".
- Graded labs will be available in the main office from Tanya by tomorrow morning (Tuesday).

Updated Friday, April 11 @~4pm (I don't expect to make any more major updates; complete solutions have now been provided as well as hopefully complete directions for the final exam)

Date: Thursday, April 17

Time: 9am-noon

Rooms: E2-1303A (seating for last names starting with 'A' to 'J') and E2-1303B (seating for last names starting with 'K' to 'Z')

Crib Sheet - both sides of single 8.5"x11" paper (or equivalently, one side each of two 8.5"x11" papers!); as for the midterm, only final equations and their titles allowed; no derivations, no solutions to problems, no text descriptions

Note:

Bring a ruler

Aids not allowed: NO calculators permitted (you really won't need one)

Material: all material Chs. 1 to 8 inclusive, all labs, all assigned problems

Recall the caveat introduced in the course outline on the first day of classes:

Given that the 3rd year SD class has exams on Monday and Tuesday, the only practical day to setup office hours is Wednesday.

If you would like to meet with one of the instructors outside these office hours, you are welcome to make an appointment via email.

Prof. Clausi: Wednesday 9-10am

Yanmin: Wednesday noon-1pm

Ossama: Wednesday 2-3pm

(Note: the above substitute for the office hours that were used during the term)

The final exam will test both your understanding of the theory and your ability to solve quantitative problems. For example, a question with a theoretical basis might ask you to compare the similarities/differences with respect to the Perceptron and MSE algorithms and a quantitative problem might involve applying one or the other technique (or both!) to a given set of data. The midterm exam was all quantitative, but I also want to evaluate your understanding of the theory on the final exam. Most of the exam will be quantitative, but I wanted to make sure you were prepared for theory-based questions. My recommendations are that you (a) develop a strong understanding of the concepts in the class, course notes, and labs and (b) learn the methdology necessary to complete the assigned problems.

*** MIDTERM EXAMINATION ***

Date: Wednesday, February 12

Time: 1:30 - 3:30pm

Aids: crib sheet (only final equations and their titles allowed; no derivations, no solutions to problems, no text descriptions); bring a ruler

Note: crib sheets will be submitted with your exam solutions; these will be returned to you

Aids not allowed: NO calculators permitted (you really won't need one)

Material: all material Chs. 1 to 4 inclusive

Office Hours in Preparation for the Midterm

Monday: Prof. Clausi, 2:30-3:30pm (DC-2611)

Monday: Ossama, 6:00-7:00pm, (in the classroom)

Tuesday: Ossama, 12:30-1:30pm (DC-2630)

Tuesday: Yanmin, 2:30-3:30pm (DC-2622) (Yanmin will not have an office hour on Thursday)

Midterm Summary Solutions

***********************************************************************************************************

On this webpage, I will be providing a host of material for this course. Note that some of the material will be added as the course progresses.

Course Outline

TA Contact Information and Office Hours:

- Ossama Elbadawy (DC-2630, x3693, ossama@watfast): Tuesdays 12:30-1:30pm
- Yanmin Sun (DC-2622, x3691, y8sun@engmail): Thursdays 1:30-2:30pm

Tutorial Schedule (will be posted on a week by week basis):

- Week 1 (January 8): Hamming Advice on Learning
- Week 2 (January 15): Problems on Statistics
- Week 3 (January 22): Matlab Tutorial
- Week 4 (January 29): Problems on Ch. 3
- Week 5 (February 5): Lab #1 Prep and Questions (Yanmin)
- Week 6 (February 12): Midterm
- Week 7 (February 19): Return Midterm
- Week 8 (February 26): Lab #2 Prep (Ossama)
- Week 9 (March 5): Problem 5.4 in course notes (Yanmin)
- Week 10 (March 12): Problem 6.4 in course notes (Ossama)
- Week 11 (March 19): Lab #3 Prep (Yanmin)

Suggested Homework Problems (will be posted on a chapter by chapter basis):

- Chapter 2: 1,3,5,6,7
- Chapter 3: 1,2,8,11,12,14
- Chapter 4: 4.1abc, 4.2, 4.3, 4.7abc, 4.8abcd (note typo in 4.8; should be '3/2' not '2/3')
- Chapter 5: 1, 4
- Chapter 6: 1, 3, 4
- Chapter 7: 1 (covered in class), 2, 3, 4
- Chapter 8: 1 (use normalized eigenvectors) , 2 (fix:
S1 = S2 = [4 0; 0 1]), 3, 4 (4(d) - sketch the 2-d MAP)

Solutions to Homework Problems (will be posted on a chapter by chapter basis):

Note that these solutions include the final answers and some hints. Solutions to all problems found in the course notes have been provided, but please note the "Suggested Homework Problems" indicated above.

- Questions 3.1 - 3.4
- Questions 3.5 - 3.6

- Questions 3.7 - 3.8
- Questions 3.9 - 3.10
- Question 3.11
- Question 3.12
- Question 3.13
- Question
3.14

- Questions 4.1 - 4.2
- Questions 4.3, 4.7
- Question 4.4
- Question 4.5
- Question 4.6

- Question 4.8
- Question 4.9
- Question 4.10

- Questions 6.1, 6.2, 6.4ab
- Note: for 6.4b, you have to demonstrate that the matrix
produced by Y'Y is not full rank; you can do this by determining that
for any real alpha, the determinant will not be zero

- Question 6.4c (part 1)
- Question 6.4c
(part 2)

- Question 7.1 - completed in class
- Question 7.2a
Question 7.2b

- Question 7.3
- Question 7.4

Previous Exams

2002 Midterm: Questions Page 1 Questions Page 2 Solutions

2001 Final Exam

Reference Material

If students would like some alternate references for course material and for additional problems to work on, I would suggest two books:

Duda, Hart & Stork (Q327.D83, On Reserve in DC Library) Pattern Classification and Scene Analysis

(There are two editions; the second is quite new, and is an excellent book. Any student seriously interested in pattern recognition could consider purchasing this book.)

Schalkoff (Q327.S27, On Reserve in DC Library) Pattern Recognition : Statistical, Structural, and Neural Approaches

(An excellent book; not as comprehensive as Duda & Hart, but more readable.)

Topic |
SD372 Course Notes |
Schalkoff |
Duda, Hart, &
Stork |

Introduction |
Ch.1 |
Ch. 1 |
Ch. 1 |

Statistics Background |
Ch. 2 |
App. 1,2 |
App. A |

Distance Classification |
Ch. 3 |
||

Statistical Classification |
Ch. 4 |
Ch. 2 |
Ch. 2 |

Parameter Estimation |
Ch. 5.1 |
Ch. 3 (p. 58-70) |
Ch. 3 |

NonParametric Estimation |
Ch. 5.2 |
Ch. 3 (p. 70-82) |
Ch. 4 |

Linear Discriminants |
Ch. 6 |
Ch. 4 |
Ch. 5 |

Unlabeled Clustering |
Ch. 7 |
Ch. 5 |
Ch. 6 |

Feature Selection |
Ch. 8 |
Ch. 1.3 |

Lab Material

Ossama has put together a Matlab summary for your use. He will speak to this during the tutorial on January 22.

Here's the pdf for Lab#1.

An ellipse drawing program is available for you here. This will come in handy for the first lab.

Here's the pdf for Lab#2.

Section 2 requires the following dataset. This section involves generating 1-d pdf estimates.

Section 3 requires the following dataset. Section 3 is an extension of Section 2 - here, 2-d pdf estimates are determined. It's not that much harder!

Prior to starting Section 3, take a look at how the data is dispersed in this image.

A means of generating the 2-D Parzen routine is provided for you. Use this matlab script.

****************************************************************************************************

Hints for Lab 2 from Ossama:

Hi Prof Clausi,

Here are the lab 2 tips:

* If you will use the parzen2.m function:

Whenever you call parzen2.m, specify the range for the second parameter to parzen2.m, and don't just pass it the resolution. I.e., always pass a vector as the second parameter to parzen2.m, and not a scalar.

Let's all use a [0 0 500 500] range (if you will use parzen2.m).

For example, if your resolution is x, always pass [x 0 0 500 500] as the second parameter to parzen2.m

* The Matlab interp2.m might be useful (will provide just an alternate solution, it won't simplify coding, and you don't have to use it).

* Note that Matlab can do the following:

if A, B, and C are three matrices as follows:

A =

1 2 3

4 5 6

7 8 9

B =

0 0 0

7 8 9

0 0 0

C =

0 0 0

0 0 0

10 11 12

>> D = A > B & A > C

D =

1 1 1

0 0 0

0 0 0

* The error computation should be done in code

*****************************************************************************************************

Officially, this lab is due April 1. However, we will accept late submissions without penalty up to and including April 4. Labs submitted after this date will not be accepted.

Here's the pdf for Lab #3.

The zip file with the images and M-files required for Lab #3 is here.

Here are the images we will be using, but please use the version in the zip file above - these are just setup so you can look at them all at the same time ...

Cloth | Cotton | Grass | Pigskin | Wood |

Cork | Paper | Stone | Raiffa | Face |