[PDF][PDF] Fairlearn: A toolkit for assessing and improving fairness in AI

S Bird, M Dudík, R Edgar, B Horn, R Lutz… - Microsoft, Tech. Rep …, 2020 - microsoft.com
S Bird, M Dudík, R Edgar, B Horn, R Lutz, V Milan, M Sameki, H Wallach, K Walker
Microsoft, Tech. Rep. MSR-TR-2020-32, 2020microsoft.com
We introduce Fairlearn, an open source toolkit that empowers data scientists and
developers to assess and improve the fairness of their AI systems. Fairlearn has two
components: an interactive visualization dashboard and unfairness mitigation algorithms.
These components are designed to help with navigating trade-offs between fairness and
model performance. We emphasize that prioritizing fairness in AI systems is a sociotechnical
challenge. Because there are many complex sources of unfairness—some societal and …
Summary
We introduce Fairlearn, an open source toolkit that empowers data scientists and developers to assess and improve the fairness of their AI systems. Fairlearn has two components: an interactive visualization dashboard and unfairness mitigation algorithms. These components are designed to help with navigating trade-offs between fairness and model performance. We emphasize that prioritizing fairness in AI systems is a sociotechnical challenge. Because there are many complex sources of unfairness—some societal and some technical—it is not possible to fully “debias” a system or to guarantee fairness; the goal is to mitigate fairness-related harms as much as possible. As Fairlearn grows to include additional fairness metrics, unfairness mitigation algorithms, and visualization capabilities, we hope that it will be shaped by a diverse community of stakeholders, ranging from data scientists, developers, and business decision makers to the people whose lives may be affected by the predictions of AI systems.
microsoft.com