What "learning" means for a machine

Traditional software follows rules that a programmer writes: if the email contains these words, mark it as spam. Machine learning turns this around. Instead of writing the rules, you give the computer many examples — emails already marked spam or not spam — and an algorithm finds the patterns that separate them.

The result is a model: a function that takes an input it has never seen and produces a prediction. Under the hood a model is usually a large set of numbers (its parameters) that were adjusted, a little at a time, until its predictions on the examples were as good as possible. That adjustment process is training.

Three things decide how good a model can be:

• The data. A model can only learn patterns that are present in its examples. If every spam email in the training set is in English, it will know nothing about spam in Spanish.
• The task. What exactly is being predicted, and from what?
• The evaluation. How we check that the model works on new data, not just on the examples it studied.

Machine learning is powerful when the rules are too complicated to write by hand — recognising a face, transcribing speech, ranking search results — and when you have plenty of good examples. It is a poor fit when you need guaranteed behaviour, when the data is scarce, or when a simple rule would do the job.

The rest of this course looks at each of those three ingredients in turn.