top of page
This page contains drafts of several chapters from our book, along with various other sections.
TABLE OF CONTENTS. [download pdf]
PREFACE. [download pdf]
CHAPTER 1. Introduction [download pdf]
How does machine learning work, and what can currently we do with it? Here we discuss the basic building blocks of machine learning and show how they all fit together to solve real problems.
CHAPTER 3. Knowledge-driven regression [download pdf]
In this Chapter we describe the regression problem, as well as the notion of feature design for regression, in particular focusing on rare low dimensional instances, and end by describing the L2 regularizer and its use with nonconvex learning problems.
CHAPTER 4. Knowledge-driven classification [download pdf]
Here the fundamental model for two-class classification, the perceptron, is introduced along with two equally effective relatives: logistic regression and support vector machines. Muticlass classification is then detailed, as well as feature design for classification with a focus on commonly used histogram-based features.
CHAPTER 5. Automatic feature design for regression
Rarely can we design strongly performing features for the general regression problem by completely relying on our understanding of a given dataset. In this Chapter we describe tools for automatically designing proper features for the general regression problem, without the explicit incorporation of human knowledge gained from e.g., visualization of the data, philisophical reflection, or domain expertise.
CHAPTER 6. Automatic feature design for classification
Here we mirror closely the exposition given in the previous Chapter on regression, beginning with the approximation of the underlying data generating function itself by bases of features, to finally describing cross-validation in the context of classification. In short we will see that all of the tools from the previous Chapter can be applied to the automatic design of features for the problem of classification as well.
CHAPTER 2. Fundamentals of numerical optimization [download pdf]
Topics include calculus defined optimality, as well as gradient descent and Newton’s method algorithms. Discussing these essential tools first will enable us to immediately and effectively deal with all of the formal learning problems we will see throughout the entirety of the text.
CHAPTER 7. Kernels, backpropagation, and regularized cross-validation
Here the notion of kernels is introduced permitting more graceful use of fixed bases to problems with vector-valued input. Next the backpropogation algorithm (also known as gradient descent) for learning with a neural network basis is detailed.
CHAPTER 8. Advanced gradient schemes
Here we introduce advanced gradient methods, including step length rules and the stochastic gradient descent method, which extend the standard gradient descent scheme first described in Chapter 2. These techniques help us deal with very large datasets directly by making use of more efficient algorithms.
CHAPTER 9. Dimension reduction techniques [download pdf]
Here we describe general techniques for significantly reducing the size of datasets prior to performing regression or classification. This includes data subsampling, K-means clustering, and Principal Component Analysis. Recommender systems are also detailed as a variant of these methods.
APPENDICES. Matrix algebra and calculus review [download pdf]
In these appendices we review fundamental vector and matrix operations, as well as elements of calculus including taking scalar and vector derivatives.
bottom of page