Hamza Waseem Naser

A filter can be 90% accurate and miss every useful item

A fictional binary-classification exercise: a collection has 100 items. Ten are relevant to a query and ninety are not. A filter labels every item “not relevant.”

Correct decisions: 90. Accuracy: 90/100 = 90%.
Relevant items recovered: 0 out of 10. Recall for the relevant class: 0/(0 + 10) = 0.

The accuracy calculation is correct. It is simply insufficient for understanding whether this filter helps someone find useful items.

When comparing a proposed improvement, show the counts: relevant items found, relevant items missed, and irrelevant items surfaced. Then discuss what each kind of error costs in this particular task. Do not quietly switch the target class or the averaging method between comparisons.

This is a toy example, not a measurement of a deployed search system or of Traitra. It shows why a score needs its task and error counts beside it.

Definitions: scikit-learn accuracy_score and recall_score documentation:
https://scikit-learn.org/stable/modules/generated/sklearn.metrics.accuracy_score.html
https://scikit-learn.org/stable/modules/generated/sklearn.metrics.recall_score.html

Which error would matter more for your fictional search task, and what information would you need before choosing?