Section outline
-
In a multi-class context, the confusion matrix expands to include a row and a column for each category, plus a "background" dimension (corresponfing to the previous negative category) representing the absence of an object. The rows indicate predictions, and the columns represent the ground truth.
The main diagonal retains its significance as correct predictions. Classification errors now occupy all off-diagonal cells. The "background" column lists false detections, while the "background" row identifies real objects that were not detected.
To calculate performance metrics (precision, recall, F1 score) and determine the optimal confidence threshold, it is necessary to decompose this global matrix. The analysis is conducted using a "one-vs-all" approach, a 2x2 binary matrix is "generated" for each class individually, isolating the true positives, false positives and false negatives specific to that category. As previously noted, true negatives are not required for this analysis.
Let us examine how to determine the three metrics, once again using the "Penny" class as an example.
For a given class (e.g. : "penny"), the metrics are defined from the matrix as follows :
- True positives (TP) : the single cell at the intersection of the class's row and column (the diagonal).
- False positives (FP) : the sum of all other cells in the class's row (predicted as the class but belonging to another).
- False negatives (FN) : the sum of all other cells in the class's column (belonging to the class but predicted as another).
These values enable the calculation of precision and recall for a specific confidence threshold. Since each confusion matrix corresponds to a unique threshold, repeating this calculation across all possible thresholds generates the performance curves. This iterative process is repeated for each class, producing individual curves. A summary curve, termed "mean" (represented in bold on the graphs), is then derived from the aggregation of all class-specific curves, offering a global view of the model's performance.
For reference, the optimal confidence threshold value, corresponding to the x-coordinate of the F1 curve's maximum, is directly indicated in the legend, immediately following the "all classes" mention.
NORMALIZED MULTI-CLASS CONFUSION MATRIX
Finally, there is the normalized confusion matrix. It provides an overall view because instead of raw counts, it shows percentages.
How to read it?
The normalized confusion matrix shows detection percentages for each class, so it is read column by column. For example, for “Penny” from top to bottom:
- 1% were predicted as “Dime”
- 5% were confused with “Nickel”
- 82% of “Penny” were correctly recognized
- 12% were not detected at all
The total sums to 100%, so everything is consistent.