Intuitive Explanation of Model Evaluation metrics

Let's say we have built a model for predicting whether a person has cancer or not.

Model Should predict, 1 if a person has cancer and 0 if a person doesn't have cancer

Let's clear some of the terminologies,

• True Positive (TP): Model predicted person has cancer and person do have cancer.
• False Positive (FP): Model predicted person has cancer but a person doesn't have cancer.
• True Negative (TN): Model predicted person don't have cancer and person doesn't have cancer
• False Negative (FN): Model predicted person doesn't have cancer but a person does have cancer.

Let's understand each of these metrics intuitively,

Accuracy

Accuracy = (TP+TN)/(TP+FP+TN+FN)

If Accuracy = 80%

It means, If the model makes predictions for 100 people, whether they have cancer or not, 80 of the predictions are correct.

Recall

Recall = TP/(TP+FN)

The recall is the model's ability to find relevant cases.

If Recall = 80%

In your dataset, if 100 people have cancer then the model can identify 80 people and will miss 20 people who have cancer

Precision

Precision is the model's ability to find only relevant case

Precision = TP/(TP+FP)

If Precision = 80%

It means, If the model has said 100 people have cancer only 80 out of them have cancer and 20 are false positive.

which metrics we should use? depends on the use case.

In the above case, Recall will be preferred since we don't want to miss anyone who has cancer.

Why accuracy is not a good metric?

If your data is an imbalance, means if it has a representation of one class is higher than other Imagine we have a total hundred patines records. where out of 100, 5 people have cancer and 95 people don't have cancer. If we use accuracy as an evaluation metric, the model can predict every patient as the non-cancer patients still will have 95% accuracy.

Is there any metric that balances identifying positive cases and also not giving any false alarms like false positives?

We can use AUROC which will choose the model which has a high True Positive Rate and Low False Positive Rate.

Photo by Javier Allegue Barros on Unsplash