Chapter 1

	Figure 1.1: Classification learning of handwritten digits. (demo.R,mnist100.dat)
	Figure 1.2: Preference learning of Web pages.
	Figure 1.3: Function learning in action (linear, cubic, 10th degree). (demo.R)
	Figure 1.4: Clustering of 150 training points, Probability density of the estimated mixture model. (demo.R)
	Figure 1.5: The first 49 digits, The 49 images in a data matrix. (demo.R, mnist100.dat)
	Figure 1.6: Classification of three new images. (demo.R)

Chapter 2

	Figure 2.1: Hypothesis space, Feature space. (grandcircle.m, plane.m, plot_hypothesis_space.m, plot_data_space.m)
	Figure 2.2: A geometrical picture of the update step in the PLA.
	Figure 2.3: Mapping of the unit square, Nine different decision surfaces. (feature.R)
	Figure 2.4: Real valued function for varying values of σ (σ=0.5, σ=0.7, σ=1.0, σ=2.0). (density.m, demo.m)
	Figure 2.5: Intensity plots of normalized Gram matrices when applying string kernels (bow kernel, subsequence kernel, substring kernel). (rbf_demo.R, strings.R, strings.c, data.tar)
	Figure 2.6: Geometrical margin of a plane (large margin, small margin). (demo.R)
	Figure 2.7: Approximation to the Heaviside step function. (loss.R)
	Figure 2.8: Largest inscribable ball in version space (3D, from top). (bayes_set.m, circle.m, grandcircle.m, plane.m, sphead.m)

Chapter 3

	Figure 3.1: Effect of evidence maximization. (demo.R)
	Figure 3.2: Gaussian process evidence maximization for a simple regression problem (log-evidence, data fit, noise fit). (demo.R, rbf_demo.R, train.dat)
	Figure 3.3: A function sampled from the ARD prior (fast variations, slow variations). (demo.R, rbf_demo.R)
	Figure 3.4: Latent variable model for classification. (demo.R, rbf_demo.R, train.dat).
	Figure 3.5: Approximation to the zero-one loss by the sigmoidal loss, Likelihood model induced by the hinge loss. (loss.R).
	Figure 3.6: Marginalized log-prior densities (n=1, n=2). (demo.R).
	Figure 3.7: Five samples obtained by playing billiards on the sphere (3-D,
	2-D). (bpm.m, arc.m, grandcircle.m, plane.m, pol2car.m).
	Figure 3.8: Fisher discriminant estimated from 80 data points, a geometrical interpretation of the Fisher discriminant. (demo.R).

Chapter 4

	Figure 4.1: True densities underlying the data in Example 4.2.
	Figure 4.2: Growth of the complexity as a function of the VC dimension (large scale, fine scale),(demo.R)
	Figure 4.3: Curse of dimensionality (n=1,n=2,n=3).
	Figure 4.4: Structural risk minimization in action. (demo.R)
	Figure 4.5: 20 real-valued functions, a cover for the function class. (demo.R)
	Figure 4.6: Relation between the real-valued output of a cover element and the real-valued of the covered output.
	Figure 4.7: Two points on the real line, γ-covering.
	Figure 4.8: Minimal training sample size. (demo.R).

Chapter 5

	Figure 5.1: Expected risk of classifiers learned by a support vector machine with and without normalization (thyroid, sonar). (normalise.R, Linear toolbox, Sonar dataset, Thyroid dataset).
	Figure 5.2: The clipped linear soft margin loss. (loss.R).

Appendix A

	Figure A.1: Geometrical proof of Scheffès theorem.
	Figure A.2: Probability mass functions (Bernoulli, Binomial, Possion, Uniform). (demo.R).
	Figure A.3: Probability densities (Gaussian, Gamma, Exponential, Beta). (demo.R).
	Figure A.4: Densities of the multidimensional normal distribution (Isotrpic covariance, general covariance). (demo.R).
	Figure A.5: ε-covering and ε-packing.

Appendix C

	Figure C.1: Graphical illustration of the main step in the basic lemma.
	Figure C.2: Counting swappings (before, after).
	Figure C.3: Counting swappings (easy, difficult).
	Figure C.4: Geometrical proof of Lemma C.12.
	Figure C.5: Comparison of the volume ratio bound and the exact value (n=10, n=100). (demo.R, volratio.c)

	Full TAR archive (contains both example datasets and source files).
	Example dataset.
	Source files: linear.R, kernels.R, kernels.c, bayes.c, kperc.c, pr_loqo.c, pr_loqo.h.

	exmp.bayes and exmp.bayes2 for Bayes point machines.
	exmp.svm for support vector machines.
	exmp.nu.svm for υ-support vector learning.
	exmp.GP.class for Gaussian process classification.
	exmp.GP.regress for Gaussian process regression.
	exmp.RVM.class for classification learning with relevance vector machines.
	exmp.RVM.regress for regression estimation with relevance vector machines.
	exmp.fisher for Fisher discriminants.
	exmp.perc for perceptron learning.

R is a statistical software package which is under the GNU Public License and publicly available at http://www.r-project.org/. This link contains pre-compiled binaries for most operating systems. The language R is very similar to the language S  which is used in the commercially available package S-PLUS developed and distributed by Insightful. Documentation for this language can be found at http://cran.r-project.org/manuals.html.

MatLab is a powerful and widely used software for numerical analysis and visualisation of scientific data (see http://www.mathworks.com/products/matlab for more information). MatLab is distributed by MathWorks Inc.).

THIS SOFTWARE IS FREE ONLY FOR NON-COMMERCIAL USE. IT MUST NOT BE MODIFIED AND DISTRIBUTED WITHOUT PRIOR PERMISSION OF THE AUTHOR. THIS SOFTWARE IS PROVIDED BY  "AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.