Precision and accuracy, bias and variance

Precision and accuracy, bias and variance

Eleven years ago, my boss at Yandex told me that reading machine learning books in Russian was a bad idea. If you did, you wouldn't know the right terms to search for online. So, for me, the language of machine learning is English. It is more or less OK. But not always. And I'm not talking (yet) about the beautiful soup of binary classification metrics. Today, let's talk about two very simple things. Precision and accuracy.

Intuitively both are about not being wrong. For a long time, that was enough for me. I memorized that accuracy is the fraction of correct predictions. Precision is the fraction of actually positive objects out of all we predicted as the positive. Suddenly my son showed me his schoolbook on statistics, where everything was explained. Precision is how close to each other your predictions are. Pictures in the book explained precision as distance between your shots and accuracy - how close your shots are to the bull's eye.

Codex is not happy with my vague approach. If you squint, you can see that this precision-accuracy pair looks resembles two quite challenging ML concepts: variance and bias. To be more precise, the relationships run in opposite directions: low precision corresponds to high variance, while high bias tends to reduce accuracy.