Dissecting Racial Bias in an Algorithm that Guides Health Decisions for 70 Million People
Ziad Obermeyer and Sendhil Mullainathan. 2019. Dissecting Racial Bias in an Algorithm that Guides Health Decisions for 70 Million People. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT* '19). Association for Computing Machinery, New York, NY, USA, 89. DOI:https://doi.org/10.1145/3287560.3287593
The authors expose racial bias in a widely used health risk algorithm, where Black patients with similar risk scores as white patients actually have far more
chronic health conditions than their white counterparts. They identify disparate risk scores across a range of medical conditions. The outcome is that white patients with the same health as Black patients are
significantly more likely to be enrolled in care management programs. They highlight that the algorithm is optimized on cost, and from this perspective, appears to be unbiased. But when applied to the actual construct of health,
it becomes racially biased. Because Black individuals cost less due social factors on healthcare, the algorithm correctly predicts costs across racial groups. The bias arises in the choice to use cost as a proxy for health in measurement. Thus, healthcare algorithms optimized for
cost amplify fundamental social biases in healthcare that prioritize finances over health.