Recall

Recall, also known as sensitivity or true positive rate, is a fundamental classification metric used to evaluate how well a model identifies positive instances. Specifically, recall measures the proportion of actual positive cases that the model correctly detects. In other words, it answers the question: "Of all the true positives, how many did the model successfully capture?"

Recall is particularly important in scenarios where missing a positive case is costly or dangerous. For example in disease diagnosis, fraud detection, or security screening, because it focuses on minimizing false negatives.

Definition and Intuition

Recall (also called sensitivity or true positive rate) measures the ability of a classification model to identify all relevant positive instances in the dataset. It is defined as the ratio of correctly predicted positive observations to all actual positives.

Mathematically, recall focuses on minimizing false negatives. Cases where the model fails to identify positive instances.

Model Performance Insights

High Recall (closer to 1)

Indicates the model successfully detects most of the actual positive cases, minimizing missed detections. This is crucial when missing a positive case has serious consequences.

Low Recall (closer to 0)

Implies many positive cases are missed by the model, leading to false negatives. This situation can be risky in critical applications like medical screening or fraud detection.

Trade-offs

Improving recall often involves accepting more false positives (lower precision). Hence, recall should be considered alongside precision to understand the balance between capturing positives and avoiding false alarms.

Core Concepts

Recall measures the proportion of all true positive cases that the model successfully identifies.

True Positives (TP)

The number of positive cases correctly identified by the model.

False Negatives (FN)

The number of positive cases missed by the model.

Recall emphasizes coverage of the positive class, ensuring as few actual positives as possible are overlooked.

Mathematical Formulation

Recall is mathematically defined as the ratio of correctly predicted positive observations (true positives) to all actual positive observations. It measures how effectively a model captures the positive class.

The formula for recall is:

$$\text{Recall} = \frac{TP}{TP + FN}$$

TP is True Positives (correctly predicted positive cases).
FN is False Negatives (positive cases missed by the model).

Calculation Procedure

Identify True Positives (TP)

Count the number of positive instances that the model predicted correctly.
Identify False Negatives (FN)

Count the number of positive instances the model failed to predict (i.e., actual positives predicted as negative).
Compute Recall

Calculate recall using the formula:

$$\text{Recall} = \frac{TP}{TP + FN}$$

The result is a value between 0 and 1, where higher values indicate better detection of positive cases.

Example

Consider a binary classification problem where we have the following confusion matrix:

	Predicted Positive	Predicted Negative
Actual Positive	80	20
Actual Negative	10	90

True Positives ($TP$) = 80
False Negatives ($FN$) = 20

Using the recall formula:

$$\text{Recall} = \frac{80}{80 + 20} = \frac{80}{100} = 0.8$$

The model correctly identifies 80% of the actual positive cases, but misses 20%. This means the model has relatively good sensitivity in detecting positives.

Properties and Behavior

Typical Value Ranges

Recall values range between 0 and 1.

Recall = 1:

The model correctly identifies all actual positive cases with no misses (no false negatives). This is ideal but often difficult to achieve in practice.

Recall = 0:

The model fails to identify any positive cases, missing all of them.

Values closer to 1 indicate better sensitivity and coverage of the positive class.

In many real-world scenarios, recall balances against precision. A very high recall may come at the cost of low precision, meaning more false positives.

Sensitivity to Outliers or Noise

Recall is generally not directly sensitive to outliers because it only considers counts of true positives and false negatives. but if noise or mislabeled data causes positive instances to be incorrectly labeled or predicted, recall will be affected, as it depends on accurate positive identification. Class imbalance or rare positive classes can also impact recall, as a small number of missed positives greatly reduce recall.

Differentiability and Role in Optimization

Recall is a non-differentiable metric since it relies on discrete counts of classification outcomes (TP and FN) rather than continuous prediction scores. Recall is not typically used as a direct loss function for training models. Instead, surrogate differentiable losses (e.g., cross-entropy loss) are minimized during training. Recall is calculated after model training to evaluate how well the model detects positive cases.

Assumptions and Limitations

Binary or multilabel focus:

Recall is most straightforward in binary classification. In multiclass problems, recall can be computed per class (class-wise recall).

Ignores false positives:

Recall does not consider false positives, so a model can have high recall but poor precision.

Class imbalance sensitivity:

In highly imbalanced datasets, recall alone may be misleading because a model could trivially achieve high recall by predicting most instances as positive.

Does not capture overall accuracy:

Recall focuses only on the positive class, so it should be interpreted in conjunction with other metrics like precision, F1-score, or accuracy.

Overall, recall is crucial when missing a positive case is costly or dangerous, but it should be evaluated alongside other metrics to get a full picture of model performance.

Code Example

fn calculate_recall(actual: &[bool], predicted: &[bool]) -> f64 {
    if actual.len() != predicted.len() {
        panic!("Length of actual and predicted values must be the same.");
    }

    let mut true_positives = 0;
    let mut false_negatives = 0;

    for (&a, &p) in actual.iter().zip(predicted.iter()) {
        if a {
            if p {
                true_positives += 1;
            } else {
                false_negatives += 1;
            }
        }
    }

    if true_positives + false_negatives == 0 {
        // No positive cases in actual labels; recall undefined, return 0.0 by convention
        return 0.0;
    }

    true_positives as f64 / (true_positives + false_negatives) as f64
}

fn main() {
    // Example data: actual and predicted labels for a binary classification
    let actual_labels = vec![true, false, true, true, false, true, false];
    let predicted_labels = vec![true, false, false, true, false, true, true];

    let recall = calculate_recall(&actual_labels, &predicted_labels);

    println!("Recall: {:.4}", recall);
}

Explanation

Input Validation:

Ensures the actual and predicted slices have the same length.

Counting true positives (TP):

Iterates over paired elements, counting instances where both actual and predicted are true.

Counting false negatives (FN):

Counts cases where the actual is true but predicted is false.

Recall calculation:

$$\text{Recall} = \frac{TP}{TP + FN}$$

Edge case:

If there are no actual positive cases (TP + FN == 0), recall is conventionally set to 0.0.

Output

Recall: 0.7500

The model correctly identified 75% of the actual positive cases.

Alternative Metrics

While Recall focuses on capturing how many of the actual positive cases the model identifies, it does not consider how many of its positive predictions are correct. Depending on the problem context, other metrics may be more suitable or useful alongside Recall.

Precision

Precision (also known as Positive Predictive Value) measures the proportion of positive predictions that were actually correct.

$$\text{Precision} = \frac{TP}{TP + FP}$$

High precision means that when the model predicts a positive class, it's usually right.
Especially important in cases where false positives are costly (e.g., spam filters, fraud detection).

"Out of all the samples predicted as positive, how many are truly positive?"

👉 A detailed explanation of Precision can be found in the section: Precision

F1 Score

The F1 Score is the harmonic mean of precision and recall. It provides a single score that balances both concerns, especially when we need to avoid both false positives and false negatives.

$$\text{F1} = 2 \cdot \frac{\text{Precision} \cdot \text{Recall}}{\text{Precision} + \text{Recall}}$$

Best used when class distribution is uneven or when both types of error matter.
Ranges from 0 (worst) to 1 (perfect).

A low F1 indicates an imbalance between precision and recall.

👉 A detailed explanation of F1 Score can be found in the section: F1 Score

Accuracy

Accuracy measures the overall proportion of correct predictions. While easy to understand, it can be misleading in imbalanced datasets where predicting the majority class yields high scores. Use it cautiously when class distributions are skewed.

$$\text{Accuracy} = \frac{TP + TN}{TP + TN + FP + FN}$$

👉 A detailed explanation of Accuracy can be found in the section: Accuracy

Specificity (True Negative Rate)

Specificity measures how well the model identifies negative cases.

$$\text{Specificity} = \frac{TN}{TN + FP}$$

It answers: Of all actual negatives, how many did the model correctly identify?

Advantages and Disadvantages

Recall is powerful when the cost of missing positive cases is high, but it should be used in conjunction with other metrics like Precision or F1 score to get a balanced view of model performance.

✅ Advantages:

Recall directly measures the ability of a model to identify all relevant positive instances, making it essential in domains where missing a positive case has serious consequences (e.g., medical diagnosis, fraud detection).
Recall is less affected by class imbalance compared to metrics like accuracy, since it focuses only on the actual positives.
It provides a straightforward interpretation, the fraction of true positives captured out of all actual positives.

❌ Disadvantages:

Recall does not account for false positives, so a model with very high Recall might produce many incorrect positive predictions, reducing overall precision.
High Recall alone is not sufficient to judge model quality. Without considering precision, a model could simply predict positive for nearly all instances to achieve high Recall.
In problems where both false positives and false negatives matter, relying solely on Recall can lead to suboptimal models.

Conclusion

Recall is a fundamental classification metric that measures the ability of a model to identify all actual positive instances in the dataset. It is valuable in scenarios where missing a positive case carries significant consequences, such as disease detection, fraud prevention, or safety-critical systems.

While Recall offers critical insight into a model’s sensitivity to positive cases, it does not provide information about the accuracy of those positive predictions. Therefore, using Recall alone can be misleading, as a model can achieve high Recall by over-predicting positives and sacrificing precision.

For a comprehensive evaluation of classification models, Recall should be considered alongside complementary metrics like Precision and the F1 score, which balance the trade-off between capturing positives and avoiding false alarms. Together, these metrics provide a nuanced view of a model’s performance.

External resources:

Example code in Rust available on 👉 GitHub Repository

Feedback

Found this helpful? Let me know what you think or suggest improvements 👉 Contact me.