| Sumario: | Fourth, note that as well as requiring that the average risk scores be equal when the base rates are, base rate tracking also requires the converse, i.e., that when the risk scores are equal, the base rates should be too. For instance, if an algorithm assigns radically different average risk scores to two groups with the same long run expected base rates, then there is something intrinsically unfair about the way that the algorithm makes its judgments, in the sense that no algorithm with this property can be perfectly fair, regardless of the details of its social/historical context etc. Since the base rates for the two rooms in Hedden's coin flip example are equal to the average risk scores assigned to the people in those rooms, base rate tracking is trivially satisfied by the optimal predictive algorithm. In contrast, when the algorithm in Redlining 1 assigned black applicants a risk score that was higher than their white counterparts in a manner that could not be justified by a comparable disparity in their base rates, that was a case in which the algorithm's predictions were themselves intrinsically unfair, and could be identified as such on the basis of purely statistical criteria.
|
|---|