Courses:

Statistical Learning Theory and Applications >> Content Detail



Study Materials



Readings

Amazon logo When you click the Amazon logo tothe left of any citation and purchase the book (or other media) from Amazon.com, MIT OpenCourseWare will receive up to 10% of this purchase and any other purchases you make during that visit. This will not increase the cost of your purchase. Links provided are to the US Amazon site, but you can also support OCW through Amazon sites in other regions. Learn more.


Special software is required to use some of the files in this course: .ps

There is no textbook for this course. All the required information will be presented in the slides associated with each class. The books/papers listed below are useful general reference reading, especially from the theoretical viewpoint. A list of suggested readings is also provided for each class separately.

Boucheron, S., O. Bousquet, and G. Lugosi. "Theory of Classification: A Survey of Recent Advances." ESAIM: Probability and Statistics 9 (2005): 323-375. (PDF)

———. "Introduction to Statistical Learning Theory." In Advanced Lectures on Machine Learning. Lecture Notes in Artificial Intelligence 3176. Edited by O. Bousquet, U. Von Luxburg, and G. Ratsch. Heidelberg, Germany: Springer, 2004, pp. 169-207. (PDF)

Bousquet, O. "New Approaches to Statistical Learning Theory." Annals of the Institute of Statistical Mathematics 55, no. 2 (2003): 371-389. (PS)

Amazon logo Vapnik, V. The Nature of Statistical Learning Theory. New York, NY: John Wiley & Sons, 1995. ISBN: 0387987800.

Amazon logo ———. Statistical Learning Theory. New York, NY: John Wiley & Sons, 1998. ISBN: 0471030031.

Amazon logo Devroye, L., L. Gyorfi, and G. Lugosi. A Probabilistic Theory of Pattern Recognition. New York, NY: Springer, 1997. ISBN: 0387946187.

Amazon logo Cristianini, N., and J. Shawe-Taylor. Introduction To Support Vector Machines. Cambridge, UK: Cambridge University Press, 2000. ISBN: 0521780195.

Evgeniou, T., and M. Pontil, and T. Poggio. "Regularization Networks and Support Vector Machines." Advances in Computational Mathematics 13, no. 1 (2000). (PDF)

Cucker, F., and S. Smale. "On The Mathematical Foundations of Learning." Bulletin of the American Mathematical Society 39, no. 1 (2002). (PS - 1.7 MB)

Poggio, T., and S. Smale. "The Mathematics of Learning: Dealing with Data." Notices of the AMS 50, no. 5 (2003): 537-544. (PDF)

Poggio, T., R. Rifkin, S. Mukherjee, and P. Niyogi. "General Conditions for Predictivity in Learning Theory." Nature 428 (2004): 419-422. (PDF) (See also Past Performance and Future Results). (PDF)


SES #TOPICSREADINGS
1The Course at a Glance
2The Learning Problem in PerspectiveCucker, F., and S. Smale. "On The Mathematical Foundations of Learning." Bulletin of the American Mathematical Society 39, no. 1 (2002): 1-49.

Evgeniou, P., and T. Poggio. "Regularization Networks and Support Vector Machines." Advances in Computational Mathematics 13, no. 1 (2000): 1-50.

Amazon logo Vapnik, V. The Nature of Statistical Learning Theory. New York, NY: John Wiley & Sons, 1995. ISBN: 0387987800.
3Reproducing Kernel Hilbert SpacesAronszajn, N. "Theory of Reproducing Kernels." Transactions of the American Mathematical Society 68, no. 3 (1950): 337-404.

Cucker, F., and S. Smale. "On The Mathematical Foundations of Learning." Bulletin of the American Mathematical Society 39, no. 1 (2002): 1-49.

Evgeniou, P., and T. Poggio. "Regularization Networks and Support Vector Machines." Advances in Computational Mathematics 13, no. 1 (2000): 1-50.

Girosi, F. "An Equivalence Between Sparse Approximation and Support Vector Machines." Neural Computation 10, no. 6 (1998): 1455-1480.

Wahba, G. "Spline Models for Observational Data Series in Applied Mathematics." SIAM 59 (1990). Philadelphia, PA: CBMS-NSF Regional Conference Series in Applied Mathematics.
4Regression and Least-Squares ClassificationRifkin, R. "Everything Old Is New Again: A Fresh Look at Historical Approaches in Machine Learning." Ph.D. Thesis, Massachusetts Institute of Technology, 2002.

Evgeniou, P., and T. Poggio. "Regularization Networks and Support Vector Machines." Advances in Computational Mathematics 13, no. 1 (2000): 1-50.

Amazon logo Vapnik, V. The Nature of Statistical Learning Theory. New York, NY: John Wiley & Sons, 1995. ISBN: 0387987800.
5Support Vector Machines for ClassificationRifkin, R. "Everything Old Is New Again: A Fresh Look at Historical Approaches in Machine Learning." Ph.D. Thesis, Massachusetts Institute of Technology, 2002.

Evgeniou, P., and T. Poggio. "Regularization Networks and Support Vector Machines." Advances in Computational Mathematics 13, no. 1 (2000): 1-50.

Amazon logo Vapnik, V. The Nature of Statistical Learning Theory. New York, NY: John Wiley & Sons, 1995. ISBN: 0387987800.
6Manifold RegularizationBelkin, M., and P. Niyogi. "Semi-supervised Learning on Riemannian Manifolds." Machine Learning 56, no. 1-3 (2004): 209-239.

Belkin, M., P. Niyogi, and V. Sindhwani. "On Manifold Regularization." In Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics. Barbados: University College London, 2005.
7Unsupervised Learning TechniquesAmazon logo Hastie, T., R. Tibshirani, and J. Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. New York, NY: Springer-Verlag, 2001. ISBN: 0387952845.

Tenenbaum, J., Vin de Silva, and J. Langford. "A Global Geometric Framework for Nonlinear Dimensionality Reduction." Science 290, no. 5500 (2000): 2319-2323.

Donoho, G. "Hessian Eigenmaps: Locally Linear Embedding Techniques for High-dimensional Data." PNAS 100, no. 10 (2003): 5591-5596.

Belkin, M., and P. Niyogi. "Laplacian Eigenmaps for Dimensionality Reduction and Data Representation." Neural Computation 15, no. 6 (2003): 1373-1396.
8MulticlassRifkin, R., and A. Klautau. "In Defense of One-Vs-All Classification." Journal of Machine Learning Research 5 (2004): 101-141.
9RankingCaruana, R., and T. Joachims, eds. Proceedings of NIPS 2002 Workshop: Beyond Classification and Regression: Learning Rankings, Preferences, Equality Predicates, and Other Structures. Cornell, NY: Cornell University, 2002.

Saul, L., Y. Weiss, and L. Bottou, eds. Proceedings of the NIPS 2005 Workshop: Learning to Rank. Whistler, BC: Cornell University, 2005.
10Boosting and BaggingFreund, Y., and R. E. Schapire. "A Short Introduction to Boosting." Journal of Japanese Society for Artificial Intelligence 14, no. 5 (1999): 771-780.
11Computer Vision

Object Detection
12Online LearningAmazon logo Duda, R., P. Hart, and D. Stork. Pattern Classification. New York, NY: John Wiley & Sons, 2001. ISBN: 9814126020.

Smale, S., and Y. Yao. "Online Learning Algorithms." Foundations of Computational Mathematics 6, no. 2 (2006): 145-170.
13Loose Ends

Project discussions
14Generalization Bounds

Intro to Stability
Amazon logo Vapnik, V. Statistical Learning Theory. New York, NY: John Wiley & Sons, 1998. ISBN: 0471030031.

Cucker, F., and S. Smale. "On The Mathematical Foundations of Learning." Bulletin of the American Mathematical Society 39, no. 1 (2002): 1-49.

Bousquet, O., and A. Elisseeff. "Stability and Generalization." Journal of Machine Learning Research 2 (2002): 499-527.
15Stability of Tikhonov RegularizationBousquet, O., and A. Elisseeff. "Stability and Generalization." Journal of Machine Learning Research 2 (2002): 499-527.
16Uniform Convergence Over Function ClassesAmazon logo Vapnik, V. The Nature of Statistical Learning Theory. New York, NY: John Wiley & Sons, 1995. ISBN: 0387987800.

Cucker, F., and S. Smale. "On The Mathematical Foundations of Learning." Bulletin of the American Mathematical Society 39, no. 1 (2002): 1-49.

Zhou, Ding-Xuan. "Capacity of Reproducing Kernel Spaces in Learning Theory." IEEE Trans on Info Theory 49, no. 7 (2003): 1743-1752.

Vapnik, V., and A. Chervonenkis. "Necessary and Sufficient Conditions for the Uniform Convergence of the Means to Their Expectations." Probability Theory and Applications 26 (1981): 532-553.
17Uniform Convergence for Classification

VC-dimension
Alon, N., S. Ben-David, N. Cesa-Bianchi, and D. Haussler. "Scale-sensitive Dimensions, Uniform Convergence, and Learnability." Journal of the ACM 44, no. 4 (1997): 615-631.

Amazon logo Vapnik, V. The Nature of Statistical Learning Theory. New York, NY: John Wiley & Sons, 1995. ISBN: 0387987800.

Cucker, F., and S. Smale. "On The Mathematical Foundations of Learning." Bulletin of the American Mathematical Society 39, no. 1 (2002): 1-49.

Vapnik, V., and A. Chervonenkis. "Necessary and Sufficient Conditions for the Uniform Convergence of the Means to Their Expectations." Probability Theory and Applications 26 (1981): 532-553.
18NeuroscienceSerre, T. "Learning a Dictionary of Shape-components in Visual Cortex: Comparison with Neurons, Humans and Machines." PhD Thesis, Massachusetts Institute of Technology, 2006.

Serre, T., L. Wolf, and T. Poggio. "Object Recognition with Features Inspired by Visual Cortex." In Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2005). San Diego, CA: IEEE Computer Society Press, 2005.

Serre, T., M. Kouh, C. Cadieu, U. Knoblich, G. Kreiman, and T. Poggio. "A Theory of Object Recognition: Computations and Circuits in the Feedforward Path of the Ventral Stream in Primate Visual Cortex." CBCL Paper #259/AI Memo #2005-036. Cambridge, MA: Massachusetts Institute of Technology, 2005.
19Symmetrization

Rademacher Averages
20Fenchel Duality
21Speech / Audio
22Active Learning
23Morphable Models for Video
24Bioinformatics
25Project Presentations
26Project Presentations (cont.)
Math Camp 1: Functional Analysis
Math Camp 2: Probability Theory

 








© 2010-2021 OpenCollege.com, All Rights Reserved.
Open College is a service mark of AmeriCareers LLC.