CSCE 970 Homework 2

Assigned: Wednesday, March 7

Due by 5:00 p.m. on Monday, April 2

Initial report due by 5:00 p.m. on Wednesday, March 28

Total number of points: 100

1. (15 pts) Do problem 4.3 on p. 129 of Theodoridis and Koutroumbas (due April 2).

2. (15 pts) Derive the weight update equations (bottom of p. 98 of the text) for exponentiated gradient when the logistic function (Equation 4.1, p. 92) is used (due April 2).

3. (15 pts) Do problem 3.5 on p. 66 of Durbin et al. (due April 2).

4. (50 pts) Each group will implement one classifier (assigned below) and train and test it on the same six data sets from homework 1. The conditions of homework 1 also apply to this homework: You must handle both binary and multiclass data, you must vary all the relevant parameters in your learning algorihtm, and you must assess training and testing speeds and classification error. In addition, all your error rates must be reported with 95% confidence intervals. You will report your initial results in a brief report that is due by 5:00 p.m. on Wednesday, March 28. This report need not be extremely detailed, but it must be sufficient for others to completely understand your classifier's performance. These results will be posted on the course web page. You will then submit a much more detailed report by 5:00 p.m. on Monday, April 2. This report will analyze your results in much greater depth and also contrast your results (error rates and run times) with all others, again at 95% confidence levels. As with homework 1, much of your grade will depend on the quality of your report, including the level of detail and the quality and depth of the analysis. Also as with homework 1, you will receive extra credit for substantial work above and beyond what is required for this homework, e.g. innovative experiments or insightful interpretation of the results.

   Here are the groups and their assigned classifiers:

   • Ben Hollingsworth: decision tree
   • Haifeng Ji and Qingping Tao: SVM with polynomial kernel (for simplicity, you need not maximize the margin)
   • Yugandhar Vankadara and Manimozhiyan Arumugam: multi-layer neural network trained with backpropagation derived from exponentiated gradient
   • Mingtian Ni and Ruiju Yang: multi-layer neural network trained with backpropagation derived from gradient descent
   • Lai Lim: k nearest neighbor classifer
   • Jun Zhang and Jiang Li: SVM with Gaussian kernel (for simplicity, you need not maximize the margin)

   For this homework, you may refer to other implementations for ideas, but you must write your own code (do not copy someone else's!). Also, if you refer to other implementations, you must cite them.

5. (5 pts) State the approximate amount of time you spent on each problem.