CSCE 478/878 (Fall 2016) lecture slides

• Lectures 0 and 1: Administrivia and Introduction, Aug 23–25, Chapters 1–2 (pdf6up)
  Also see:
  • Domingos article on machine learning
  • Andrew Ng's Lecture 1 of his course at Stanford
  • Andrew Ng's Lecture 2 of his course at Stanford (discusses autonomous vehicle)
  • Big Think: 22 Ways Algorithms Know How You'll Behave Before You Do
  • Big Think: Listen to These Odd Pop Songs Composed by a Computer
  • Big Think: What Do a Robot's Dreams Look Like? Google Found Out
  • Google's Deep Dream Generator


• Homework 0 due Aug 30


• Lecture 2: Supervised Learning, Aug 25–30, Chapter 2 (pdf1up, pdf6up)


• Homework 1 due Sunday, September 25


• Mini-Homework 1.5 due Thursday, September 15


• Lecture 4: Experimental Design and Analysis Sep 8–15, Chapter 19 and ROC tutorial (pdf1up, pdf6up)
  Also see:
  • Tom Fawcett. An Introduction to ROC Analysis. Pattern Recognition Letters, 27:861–874, 2006.
  • ICML 04 ROC tutorial by Peter Flach
  • Thomas G. Dietterich. Approximate statistical tests for comparing supervised classification learning algorithms. Neural Computation 10(7):1895–1924.
  • Chris Drummond and Robert C. Holte Cost curves: An improved method for visualizing classifier performance. Machine Learning, 65(1):95–130, 2006.
  • Marina Sokolova and Guy Lapalme. A Systematic Analysis of Performance Measures for Classification Tasks. Information Processing & Management, 45(4):472–437, July 2009.


• Lecture 5: Artificial Neural Networks and Support Vector Machines Sep 20–Oct 6, Chapters 10, 11, and 13 (pdf1up, pdf6up)
Also see:
• Nello Cristianini and Bernhard Scholkopf. Support Vector Machines and Kernel Methods: The New Generation of Learning Machines. AI Magazine, Vol. 23, No. 3 (2002).
• An Introduction to Support Vector Machines, by Nello Cristianini and John Shawe-Taylor. Cambridge University Press, 2000. ISBN 0-521-78019-5.
• Which Parts Of The Brain Do What? from MinuteEarth


• Homework 2 due Sunday, October 16


• Mini-Homework 2.5 due Friday, October 7


• Lecture 6: Bayesian learning, Oct 6–20, Chapter 14 (pdf1up, pdf6up)
  Also see:
  • The Bayesian Trap. Veritasium.
  • Eugene Charniak. Bayesian Networks without Tears. AI Magazine, vol. 12, no. 4 (1991).
  • Nir Friedman and Daphne Koller. Learning Bayesian Networks from Data (NIPS tutorial).
  • Pedro Domingos and Michael Pazzani. Beyond independence: Conditions for the optimality of the simple bayesian classifier. In Proceedings of the 13th International Conference on Machine Learning, 1996.
  • Stuart Russell and Peter Norvig. Artificial Intelligence: A Modern Approach, Third Edition. Prentice Hall, 2009.
  • Richard E. Neapolitan. Learning Bayesian Networks. Prentice Hall, 2004.
  • Neapolitan's KDD lecture on causality
  • Russ Greiner's page of links on Bayes nets
  • Tom Dietterich's course on Probabilistic Relational Models
  • Geiger et al.'s paper on algorithms for d-separation
  • Judea Pearl. Probabilistic Reasoning in Intelligent Systems: Networks of Probabilistic Inference. Morgan Kauffmann, 1988.
  • Daphne Koller, Nir Friedman, Lise Getoor, and Ben Taskar. Graphical Models in a Nutshell.


• Homework 3 due Sunday, October 30


• Mini-Homework 3.5 due Wednesday, October 26


• Lecture 7: Bagging and Boosting, Oct 25–27, Chapter 17, Breiman paper, Freund & Schapire paper (pdf1up, pdf6up)
  Also see:
  • Leo Breiman. Bagging predictors. Technical Report 421. Dept. of Statistics, University of California-Berkeley. September 1994.
  • Yoav Freund and Robert Schapire. A short introduction to boosting. Journal of Japanese Society for Artificial Intelligence 14(5):771–780, September, 1999. [Other introductory and advanced papers are available at Freund's and Schapire's pages.]
  • Robert Schapire and Yoav Freund. Boosting: Foundations and Algorithms. MIT Press, 2012. ISBN 0-262-01718-0.
  • Nick Littlestone and Manfred K. Warmuth. The weighted majority algorithm.
  Information and Computation 108(2):212–261. February 1994.


• Lecture 8: Clustering, Oct 27–Nov 1, Chapter 7 (pdf1up, pdf6up)


• Lecture 9: Hidden Markov Models, Nov 3–15, Chapter 15 (pdf1up, pdf6up)
  Also see:
• Biological Sequence Analysis: Probabilistic Models of Proteins and Nucleic Acids, by Richard Durbin, Sean Eddy, Anders Krogh, and Graeme Mitchison. Cambridge University Press, 1998. ISBN 0-521-62971-3.
• Section 9.6 of Pattern Recognition by Sergios Theodoridis and Konstantinos Koutroumbas
• K. Sjölander, K. Karplus, M. Brown, R. Hughey, A. Krogh, I. S. Mian, and D. Haussler. Dirichlet mixtures: A method for improving detection of weak but significant protein sequence homology. Computer Applications in the Biosciences (CABIOS), 12(4):327–345, 1996.


• Mini-Homework 4.5 due Tuesday, November 15


• Lecture 10: How to give a good research talk, Nov 15–17 (pdf1up, pdf6up)
  Also see:
  • Tips on Presenting Technical Material
  • Guidelines for Presentations by Daniele Radicioni, 2008


• Lecture 11: Reinforcement learning, Nov 17–22 (pdf1up, pdf6up)
Also see:
• Gerald Tesauro. Temporal Difference Learning and TD-Gammon. Communcations of the ACM 38(3):58-68, March 1995.
• Richard Sutton and Andrew Barto. Reinforcement Learning: An Introduction. MIT Press, 1998.

Last modified 06 April 2017; please report problems to sscott.