CSCE 478/878 (Fall 2006) lecture slides

• Lectures 0 and 1: Administrivia and Introduction, Aug 22-24, Chapter 1 (pdf1up, pdf6up) Also see:
  • My slides introducing machine learning to the Future Problem Solving Program at Central Community College

• Lecture 2: Concept Learning and the General-to-Specific Ordering, Aug 29-31, Chapter 2 (pdf1up, pdf6up)
  Topic summary 1 due Thursday, Sept 7

• Homework 0 due Aug 31

• Prerequisite test due Sep 3

• Lecture 3: Decision Trees, overfitting, Occam's razor, Aug 31-Sep 12, Chapter 3 (pdf1up, pdf6up)
  Topic summary 2 due Tuesday, Sep 19 Also see:
  • C4.5: Programs for Machine Learning
  • Manuals/tutorials for C4.5

• Homework 1 Assigned Thursday, September 7; Due Sunday, September 24 at 11:59 p.m.

• Lecture 4: Artificial Neural Networks and Support Vector Machines Sep 14-Sep 26, Chapter 4, Section 7.4.4, Littlestone paper, handout from Christianini and Shawe-Taylor's book (pdf1up, pdf6up) Also see:
  • Manfred K. Warmuth, who has done much work on EG and Winnow. Many papers available on-line, including his GD/EG paper:
    • Jyrki Kivinen and Manfred K. Warmuth. Exponentiated gradient versus gradient descent for linear predictors. Information and Computation 132(1):1-64, January 1997.
  • Nick Littlestone (creator of Winnow) and some of his papers:
    • Nick Littlestone. Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm. Machine Learning 2:285-318, 1988. [The linked paper is not the final version, but a similar technical report.]
    • Nick Littlestone. Redundant noisy attributes, attribute errors, and linear threshold learning using Winnow. In Proc. 4th Annu. Workshop on Comput. Learning Theory, 147-156, 1991. Morgan Kaufmann.
  • Avrim Blum and his survey paper ``On-Line Algorithms in Machine Learning''
  • Notes from CSCE 970 on nonlinear classifiers [on the intuition behind using multiple layers to remap the input space] (pdf4up)
  • ANN growing and pruning:
    • Christopher M. Bishop. Neural Networks for Pattern Recognition. Oxford University Press, 1995. (Section 9.5)
  • SVMs:
    • Muller, K.-R., Mika, S., Ratsch, G., Tsuda, K., and Scholkopf, B. An introduction to kernel-based learning algorithms. IEEE Transactions on Neural Networks, 12(2):181-201, 2001.
    • Christopher Burges. A tutorial on support vector machines for pattern recognition
    • Nello Cristianini and John Shawe-Taylor. An Introduction to Support Vector Machines. Cambridge University Press, 2000.
    • Richard Duda, Peter Hart, and David Stork. Pattern Classification, 2nd Edition. [also see software supplements] John Wiley, 2001. (Section 5.11)
    • SVM tutorial
    • KernelMachines.org
    • Sebastian Thrun's links

• Lecture 5: Evaluating Hypotheses Oct 13-24, Chapter 5 and ROC tutorial (pdf1up, pdf6up) Topic summary 4 due Tuesday, Oct 31
  Also see:
  • ICML 04 ROC tutorial by Peter Flach
  • Thomas G. Dietterich. Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation 10(7):1895-1924.
  • Chris Drummond and Robert C. Holte Cost curves: An improved method for visualizing classifier performance. Machine Learning, 65(1):95-130, 2006.

• Fall break, Oct 17, no class

• Lecture 6: Combining Classifiers: Weighted Majority, Bagging and Boosting, Oct 26-31, Section 7.5.4, Breiman paper, Freund & Schapire paper (pdf1up, pdf6up) Topic summary 6 due Tuesday, Nov 7
  Also see:
  • Leo Breiman. Bagging predictors. Technical Report 421. Dept. of Statistics, University of California-Berkeley. September 1994.
  • Yoav Freund and Robert Schapire. A short introduction to boosting. Journal of Japanese Society for Artificial Intelligence 14(5):771-780, September, 1999. [Other introductory and advanced papers are available at Freund's and Schapire's pages.]
  • Nick Littlestone and Manfred K. Warmuth. The weighted majority algorithm. Information and Computation 108(2):212-261. February 1994.

• Lecture 7: Bayesian learning, Oct 31-Nov 9, Chapter 6 (pdf1up, pdf6up) Also see:
  • Pedro Domingos and Michael Pazzani. Beyond independence: Conditions for the optimality of the simple bayesian classifier. In Proceedings of the 13th International Conference on Machine Learning, 1996.
  • Stuart Russell and Peter Norvig. Artificial Intelligence: A Modern Approach, Second Edition. Prentice Hall, 2003.
  • Richard E. Neapolitan. Learning Bayesian Networks. Prentice Hall, 2004.

• Lecture 8: Instance-based learning, Nov 14-21, Chapter 8 (pdf1up, pdf4up) Also see:
  • Case-based reasoning (thanks to Brandon Hauff):
    • Aamodt, A. and Plaza, E. (1994). Case-Based Reasoning: Foundational Issues, Methodological Variations, and System Approaches. AI Communications, 7(i): pp 39-59.
    • CBR tutorial
    • Ralph Bergmann's site
  • David Aha, ed. Lazy Learning. Kluwer Academic Publishers, 1997.
  • Christopher M. Bishop. Neural Networks for Pattern Recognition. Oxford University Press, 1995. [RBF networks]
  • Sergios Theodoridis and Konstantinos Koutroumbas. Pattern Recognition. [distance measures between instances]

• Lecture 8.5: How to give a good research talk, Nov 21-28, (pdf1up, pdf6up)
  Also see:
  • Tips on Presenting Technical Material

• Thanksgiving holiday, Nov 22-26, no class

• Lecture 9: Students' choice, Nov 28-30:
  1. Computational Learning Theory, Sections 7.1-7.4.3, 7.5-7.5.3, 7.6 (pdf1up, pdf4up)
  2. Reinforcement learning, Chapter 13 (pdf1up, pdf4up) Also see:
     • Gerald Tesauro. Temporal Difference Learning and TD-Gammon. Communcations of the ACM 38(3):58-68, March 1995.
     • Richard Sutton and Andrew Barto. Reinforcement Learning: An Introduction. MIT Press, 1998.
  3. Genetic Algorithms, Chapter 9 (pdf1up, pdf4up)
  4. New topic: multiple-instance learning or active learning or budgeted learning

• Project presentations, Dec 5-8 (dead week)

• Project reports due Dec 8

Last modified 16 August 2011; please report problems to sscott.