Exercise 8

Part of the course Machine Learning for Materials and Chemistry.

Task 8.1: Performance curves

Write (or extend previous) code to learn the function x**2 using KRR and a Gaussian kernel. Plot the loss function as a function of training set size. Plot the average error in the minimum position and the curvature at the minimum as a function of training set size in the same plot. What do you observe? Note: you can obtain the curvature at the minimum by finite differences.

Task 8.2: Regularization

Now plot the loss function over training set size for noisy data where you add Gaussian noise to each training point. Set lambda in KRR to different values. What do you observe?

Task 8.3: Implicit knowledge

A simpler model would be to directly fit a quadratic function to the training data and extract minimum and curvature from there directly rather than building the KRR model of task 1. Implement this simpler model for the noisy data of task 2. What do you observe? What would be the result of the comparison if we had no noise?