CSCE 970 lecture slides
Entries in red do not have slides updated for
spring 2003. Refer to the the spring 2001 offering
for old copies of slides.
Lectures 0 and 1: Administrivia and Introduction, Jan 14. Theodoridis
and feature vectors, classification, supervised vs. unsupervised learning
Lecture 2: Bayesian-Based Classifiers, Jan 16-21. Theodoridis
Sections 2.1-2.4, 2.5.1, 2.5.2, 2.5.6, 2.6.
Topic summary 1 due Thursday, Feb 6
Bayesian decision theory, discriminant
functions, Bayesian classification for Gaussian distributions, estimation of
unknown pdfs, k-nearest neighbor techniques
Lecture 3: Linear Classifiers, Jan 21-28. Theodoridis Sections 3.1-3.3, 3.4.1,
3.4.2, 3.5 (skim), pages 1-19 of
Topic summary 2 due Tuesday, Feb 11
Linear discriminant functions, perceptron
algorithm, Winnow, exponentiated gradient, least squares methods
- Manfred K. Warmuth,
who has done much work on EG and Winnow. Many papers available on-line,
- Nick Littlestone (creator of Winnow) and some of his papers:
- N. Littlestone. ``Learning Quickly When Irrelevant Attributes Abound: A
New Linear-threshold Algorithm''. Machine Learning, 2:285-318, 1988.
[original Winnow paper]
- N. Littlestone. ``Redundant noisy attributes, attribute errors, and
linear threshold learning using Winnow''. In
Proc. 4th Annu. Workshop on Comput. Learning Theory,
147-156, 1991. Morgan Kaufmann.
[agnostic Winnow results]
- A. Grove, N. Littlestone, and
results for linear discriminant updates. Machine Learning 43(1-3):173-210,
[gives nice presentation of Winnow with negative weights + very general
- Avrim Blum and his survey paper
``On-Line Algorithms in Machine Learning''
- Thomas G. Dietterich
and his paper
``Solving Multiclass Learning Problems via Error-Correcting Output Codes''
Lecture 4: Nonlinear Classifiers. Theodoridis Sections 4.1-4.4, 4.6 (skip
proof), 4.7, 4.9, 4.10, 4.13-4.15, 4.17.
Topic summary 3 due Tuesday, Mar 11
2- and 3-layer perceptrons, backpropagation,
setting network size (especially pruning), Cover's theorem, RBF networks,
- ANN growing and pruning:
- Muller, K.-R., Mika, S., Ratsch, G., Tsuda, K., and Scholkopf, B.
to kernel-based learning algorithms.
IEEE Transactions on Neural Networks, 12(2):181-201,
- Christopher Burges.
tutorial on support vector machines for pattern recognition
- Nello Cristianini
and John Shawe-Taylor.
An Introduction to Support Vector Machines.
Cambridge University Press, 2000.
- Richard Duda,
Peter Hart, and David Stork.
Pattern Classification, 2nd Edition.
[also see software supplements]
John Wiley, 2001.
- SVM tutorial
- Sebastian Thrun's links
- Decision trees:
work on learning geometric patterns (a generalization
of multiple-instance learning), especially the paper based on Winnow.
Lecture 5: Hidden Markov Models. Durbin Chapter 3, Theodoridis Sections
9.1-9.4, 9.6, 9.8.
Topic summary 4 due Thursday, March 27
Markov models, the Viterbi algorithm, hidden Markov models, Baum-Welch
- R. Durbin, S. Eddy, A. Krogh, and G. Mitchison. Biological Sequence
Analysis. Cambridge University Press, 1998. [see ch. 3]
Shamir's course on computational biology [see the scribe notes on hidden Markov models]
- ISMB99 Tutorial on HMMs
I. S. Mian, and
Dirichlet mixtures: A method for improving detection of weak
but significant protein sequence homology. Computer Applications
in the Biosciences (CABIOS), Vol. 12, No. 4, Pages 327-345, 1996.
- HMM Tutorial
- Source code (specific to biological sequence analysis):
Lecture 6: System Evaluation and Combining Classifiers. Theodoridis Chapter
10, selected papers.
Topic summary 5 due Tuesday, April 8
Estimating classification error (confidence intervals, paired t tests,
cross-validation), improving performance (bagging, boosting, weighted majority).
Lecture 7: Clustering: Basic Concepts. Theodoridis Chapter 11, Sections
Applications, examples, cluster types, feature
types, proximity measures, categories of algorithms.
Lecture 8: Sequential Clustering Algorithms. Theodoridis Sections 12.3-12.6.
BSAS, MBSAS, TTSAS, estimating the number of clusters.
Lecture 9: Hierarchical Clustering Algorithms. Theodoridis Sections 13.1,
Topic summary 6 (on Lectures 7-9) due Tuesday, April 15
Agglomerative schemes (dendograms,
single link algorithm, complete link algorithm), determining the best number
Lecture 10: Clustering Algorithms Based on Cost Function Optimization.
Theodoridis Sections 14.1, 14.3.1, 14.3.6, 14.5, selected papers.
Topic summary 7 due Tuesday, April 22
Isodata algorithm, fuzzy
clustering methods (also fuzzy classification, if time permits).
Lecture 11: Clustering Tendency and Cluster Validity. Theodoridis Chapter 16.
NOTE: These are different from what was handed out in class;
two more slides are in this set.)
internal criteria, external criteria, relative criteria, validity of
individual clusters, cluster tendency.
- Section 5.3.1 ("Hypothesis Testing Basics") from the text
Special Lecture: How to Give a Good Research Talk.
Return to the CSCE 970 Home Page