In algorithms of binary classification I usually meet the terms “precision” and “recall.” I had a really hard time understanding their meaning on a basic level. In other words, how would I explain them to a five year-old? I finally found these definitions:
- Precision is how often our algorithm predicts class one and turns out to be right.
- Recall is how many objects of class one the algorithm has found at all.
Let’s use an example to illustrate it even more clearly. We want to check if our fruit is an orange and give an answer “class one” if a particular fruit is in fact an orange and “not class one” if it’s not.
We have developed some algorithm for that, and we don’t know how good this algorithm is.
Let’s say we have 10 fruits in total; three of them are oranges, seven are apples. (That we know for sure, but we’re unsure of our algorithm).
Next, we apply the algorithm and it tells us: “You have five oranges and five not oranges.” Hmm… Could have been better, huh? So here is our “confusion matrix”— we are writing down the results of our algorithm and actual results:
|algorithm says it’s an orange||algorithm says it’s not an orange|
|we can see it’s an orange||3||0|
|we can see it’s not an orange||2||5|
Precision here = 3 / (3 + 2) = algorithm says it’s an orange and it’s really an orange / (algorithm says it’s an orange) = 3/5 = 0.6.
So we can say that in 60% of cases when our algorithm reports class one (orange), it turns to be right.
Recall here = 3 / 3 + 0 = algorithm says it’s an orange and it’s really an orange / (algorithm says it’s an orange and it’s really an orange + algorithm says it’s not an orange but it is an orange) = 1
Recall here is perfect—we can say that we found 100% of our oranges.
The next step depends on our task: if we are okay with an algorithm that found oranges among apples but didn’t miss any real oranges, then we are done!