CSCE 478/878 (Fall 2001) lecture slides

Lectures 0 and 1: Administrivia and Introduction, Aug 27-29, Chapter 1 (ps1up, pdf1up, ps4up, pdf4up)

Lecture 2: Concept Learning and the General-to-Specific Ordering, Sep 5-10, Chapter 2 (ps1up, pdf1up, ps4up, pdf4up) Topic summary 1 due Monday, Sept 17

Lecture 3: Computational Learning Theory, Sep 10-19, Sections 7.1-7.4.3, 7.5-7.5.3, 7.6 (ps1up, pdf1up, ps4up, pdf4up) Topic summary 2 due Wednesday, Sept 26

Lecture 4: Decision Trees, overfitting, Occam's razor, Sep 19-26, Chapter 3 (ps1up, pdf1up, ps4up, pdf4up) Topic summary 3 due Wednesday, Oct 3
Topic summary 3 now due Friday, Oct 5 Also see:

• C4.5: Programs for Machine Learning
• Complete source for C4.5 (old version)
• Manuals/tutorials for C4.5

Lecture 5: Artificial Neural Networks Oct 1-15, Chapter 4, Section 4.4, Littlestone paper (ps1up, pdf1up, ps4up, pdf4up) Topic summary 4 due Wednesday, Oct 24
Also see:

• Manfred K. Warmuth, who has done much work on EG and Winnow. Many papers available on-line, including his GD/EG paper:
  • Jyrki Kivinen and Manfred K. Warmuth. Exponentiated gradient versus gradient descent for linear predictors. Information and Computation 132(1):1-64, January 1997.
• Nick Littlestone (creator of Winnow) and some of his papers:
  • Nick Littlestone. Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm. Machine Learning 2:285-318, 1988.
  • Nick Littlestone. Redundant noisy attributes, attribute errors, and linear threshold learning using Winnow. In Proc. 4th Annu. Workshop on Comput. Learning Theory, 147-156, 1991. Morgan Kaufmann.
• Avrim Blum and his survey paper ``On-Line Algorithms in Machine Learning''
• Notes from CSCE 970 on nonlinear classifiers [on the intuition behind using multiple layers to remap the input space] (ps4up, pdf4up)
• ANN growing and pruning:
  • Christopher M. Bishop. Neural Networks for Pattern Recognition. Oxford University Press, 1995. (Section 9.5)

Lecture 6: Evaluating Hypotheses Oct 17-24, Chapter 5 (ps1up, pdf1up, ps4up, pdf4up) Topic summary 5 due Wednesday, Oct 31
Also see:

• Thomas G. Dietterich. Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation 10(7):1895-1924.

Lecture 7: Combining Classifiers: Weighted Majority, Bagging and Boosting, Oct 29-31, Section 7.5.4, Breiman paper, Freund & Schapire paper (ps1up, pdf1up, ps4up, pdf4up) Topic summary 6 due Wednesday, Nov 7
Also see:

• Leo Breiman. Bagging predictors. Technical Report 421. Dept. of Statistics, University of California-Berkeley. September 1994.
• Yoav Freund and Robert Schapire. A short introduction to boosting. Journal of Japanese Society for Artificial Intelligence 14(5):771-780, September, 1999. [Other introductory and advanced papers are available at Freund's and Schapire's pages.]
• Nick Littlestone and Manfred K. Warmuth. The weighted majority algorithm. Information and Computation 108(2):212-261. February 1994.

Special Event: Colloquium by Pat Langley on adaptive user interfaces, scheduled for Monday, November 5, 2:30pm in CBA 25.

Lecture 8: Bayesian learning, Nov 5-Nov 14, Chapter 6 (ps1up, pdf1up, ps4up, pdf4up) Topic summary 7 due Monday, Nov 26
Also see:

• Pedro Domingos and Michael Pazzani. Beyond independence: Conditions for the optimality of the simple bayesian classifier. In Proceedings of the 13th International Conference on Machine Learning, 1996.
• Stuart Russell and Peter Norvig. Artificial Intelligence: A Modern Approach. Prentice Hall, 1995. [Good description of Bayes nets]

Lecture 9: Instance-based learning, Nov 14-26, Chapter 8 (ps1up, pdf1up, ps4up, pdf4up) Topic summary 8 tentatively due Monday, Dec 3
Also see:

• David Aha, ed. Lazy Learning. Kluwer Academic Publishers, 1997.
• Christopher M. Bishop. Neural Networks for Pattern Recognition. Oxford University Press, 1995. [RBF networks]
• Sergios Theodoridis and Konstantinos Koutroumbas. Pattern Recognition. [distance measures between instances]

Lecture 9.5: How to give a good research talk, Nov 28, Goldman slides (ps4up, pdf4up)
Also see:

• Tips on Presenting Technical Material

Lecture 10: Reinforcement learning, Nov 26-Dec 3, Chapter 13 (ps1up, pdf1up, ps4up, pdf4up) Also see:

• Gerald Tesauro. Temporal Difference Learning and TD-Gammon. Communcations of the ACM 38(3):58-68, March 1995.
• Richard Sutton and Andrew Barto. Reinforcement Learning: An Introduction. MIT Press, 1998.

Lecture 11: Genetic Algorithms, Dec 3, Chapter 9 (ps1up, pdf1up, ps4up, pdf4up)

Special lecture by Cory Strope on a multiple-instance learning algorithm, and course evaluations (ps, pdf), Dec 5

Project presentations, Dec 10-12

Special Event: Colloquium by Sally Goldman on multiple-instance learning, scheduled for Thursday, December 13, 3:30pm [refreshments in 114 Ferguson], 3:45pm [talk in 217 Ferguson] (abstract, colloquium schedule)

Last modified 16 August 2011; please report problems to sscott AT cse.