What are the criteria of good pattern recognition?

Back

Sep. 22, 2021

Knowing how to assess the performance of a pattern recognition model is highly important for a wide variety of tasks in artificial intelligence, machine learning, and computer vision. Below, I’ll discuss some of the most widely used criteria for good pattern recognition systems.

Learn from Chooch

Suppose that you’re working on a simple image classification task for pattern recognition: identifying when a photograph has a dog in it. You have 100 images of each type (dog vs. no dog) in your test dataset. The first step is to separate your model’s results into four types:

  • True positives: The model correctly predicts that an image contains a dog.
  • False positives: The model incorrectly predicts that an image contains a dog.
  • True negatives: The model correctly predicts that an image does not contain a dog.
  • False negatives: The model incorrectly predicts that an image does not contain a dog.

Once you have the figures for each of these four types, you can build a confusion matrix. Suppose your confusion matrix looks like this:

  • True positives: 98
  • False positives: 5
  • True negatives: 95
  • False negatives: 2

With this information, you can use the following metrics to assess your model’s performance:

  • Precision: the number of true positives, divided by the total number of positives that the model predicts. Here, the model’s precision for detecting dogs is 98/(98+5) = 95.1%.
  • Recall: the number of true positives, divided by the total number of positives in the original dataset. Here, the model’s recall for detecting dogs is 98/(98+2) = 98%.
  • F1 score: the harmonic mean of precision and recall, i.e. 2*(precision*recall)/(precision+recall). Here, the model’s F1 score is 2*(95.1*98)/(95.1+98) = 96.5.

In addition to precision, recall, and F1 score, another criterion used to evaluate machine learning and computer vision models is ROC-AUC. An ROC curve is used to visualize the performance of a binary classifier model, and AUC (“Area Under Curve”) is a metric for measuring how well the classifier can separate signal from noise. If AUC is equal to 1, the model can perfectly separate positive and negative classes; if it is equal to 0, the model incorrectly predicts all positives as negatives, and vice versa.

×

Oops, something went wrong.
Please try back later.

Click here Tap here to upload Upload your image
and Chooch will recognize it.
Supported formats are .jpg .jpeg .png

×