Monitoring Algorithmic Fairness Under Partial Observations
Pages 291 - 311
Abstract
As AI and machine-learned software are used increasingly for making decisions that affect humans, it is imperative that they remain fair and unbiased in their decisions. To complement design-time bias mitigation measures, runtime verification techniques have been introduced recently to monitor the algorithmic fairness of deployed systems. Previous monitoring techniques assume full observability of the states of the (unknown) monitored system. Moreover, they can monitor only fairness properties that are specified as arithmetic expressions over the probabilities of different events. In this work, we extend fairness monitoring to systems modeled as partially observed Markov chains (POMC), and to specifications containing arithmetic expressions over the expected values of numerical functions on event sequences. The only assumptions we make are that the underlying POMC is aperiodic and starts in the stationary distribution, with a bound on its mixing time being known. These assumptions enable us to estimate a given property for the entire distribution of possible executions of the monitored POMC, by observing only a single execution. Our monitors observe a long run of the system and, after each new observation, output updated PAC-estimates of how fair or biased the system is. The monitors are computationally lightweight and, using a prototype implementation, we demonstrate their effectiveness on several real-world examples.
References
[1]
Agha G and Palmskog K A survey of statistical model checking ACM Trans. Model. Comput. Simul. (TOMACS) 2018 28 1 1-39
[2]
Albarghouthi, A., D’Antoni, L., Drews, S., Nori, A.V.: FairSquare: probabilistic verification of program fairness. Proc. ACM Program. Lang. 1(OOPSLA), 1–30 (2017)
[3]
Albarghouthi, A., Vinitsky, S.: Fairness-aware programming. In: Proceedings of the Conference on Fairness, Accountability, and Transparency, pp. 211–219 (2019)
[4]
Baier C, Haverkort B, Hermanns H, and Katoen JP Model-checking algorithms for continuous-time Markov chains IEEE Trans. Softw. Eng. 2003 29 6 524-541
[5]
Baier C and Katoen JP Principles of Model Checking 2008 Cambridge MIT Press
[6]
Balunovic, M., Ruoss, A., Vechev, M.: Fair normalizing flows. In: International Conference on Learning Representations (2021)
[7]
Bartocci E et al. Bartocci E, Falcone Y, et al. Specification-based monitoring of cyber-physical systems: a survey on theory, tools and applications Lectures on Runtime Verification 2018 Cham Springer 135-175
[8]
Bartocci E and Falcone Y Lectures on Runtime Verification 2018 Cham Springer
[9]
Bartolo Burlò C, Francalanza A, Scalas A, Trubiani C, and Tuosto E Damiani F and Dardha O Towards probabilistic session-type monitoring Coordination Models and Languages 2021 Cham Springer 106-120
[10]
Bellamy, R.K., et al.: AI fairness 360: an extensible toolkit for detecting and mitigating algorithmic bias. IBM J. Res. Dev. 63(4/5), 4–1 (2019)
[11]
Bird, S., et al.: Fairlearn: a toolkit for assessing and improving fairness in AI. Microsoft, Technical report. MSR-TR-2020-32 (2020)
[12]
Chouldechova A Fair prediction with disparate impact: a study of bias in recidivism prediction instruments Big Data 2017 5 2 153-163
[13]
Clarke EM and Zuliani P Bultan T and Hsiung P-A Statistical model checking for cyber-physical systems Automated Technology for Verification and Analysis 2011 Heidelberg Springer 1-12
[14]
D’Amour, A., Srinivasan, H., Atwood, J., Baljekar, P., Sculley, D., Halpern, Y.: Fairness is not static: deeper understanding of long term fairness via simulation studies. In: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, FAT* 2020, pp. 525–534 (2020)
[15]
David A, Du D, Guldstrand Larsen K, Legay A, and Mikučionis M Brat G, Rungta N, and Venet A Optimizing control strategy using statistical model checking NASA Formal Methods 2013 Heidelberg Springer 352-367
[16]
Donzé A and Maler O Chatterjee K and Henzinger TA Robust satisfaction of temporal logic over real-valued signals Formal Modeling and Analysis of Timed Systems 2010 Heidelberg Springer 92-106
[17]
Dressel, J., Farid, H.: The accuracy, fairness, and limits of predicting recidivism. Sci. Adv. 4(1), eaao5580 (2018)
[18]
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., Zemel, R.: Fairness through awareness. In: Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, pp. 214–226 (2012)
[19]
Ensign, D., Friedler, S.A., Neville, S., Scheidegger, C., Venkatasubramanian, S.: Runaway feedback loops in predictive policing. In: Conference on Fairness, Accountability and Transparency, pp. 160–171. PMLR (2018)
[20]
Esposito, A.R., Mondelli, M.: Concentration without independence via information measures. arXiv preprint arXiv:2303.07245 (2023)
[21]
Feldman, M., Friedler, S.A., Moeller, J., Scheidegger, C., Venkatasubramanian, S.: Certifying and removing disparate impact. In: proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 259–268 (2015)
[22]
Ferrere, T., Henzinger, T.A., Kragl, B.: Monitoring event frequencies. In: 28th EACSL Annual Conference on Computer Science Logic, vol. 152 (2020)
[23]
Ferrère, T., Henzinger, T.A., Saraç, N.E.: A theory of register monitors. In: Proceedings of the 33rd Annual ACM/IEEE Symposium on Logic in Computer Science, pp. 394–403 (2018)
[24]
Finkbeiner B, Sankaranarayanan S, and Sipma H Collecting statistics over runtime executions Electron. Notes Theor. Comput. Sci. 2002 70 4 36-54
[25]
Ghosh, B., Basu, D., Meel, K.S.: Justicia: a stochastic sat approach to formally verify fairness. arXiv preprint arXiv:2009.06516 (2020)
[26]
Ghosh, B., Basu, D., Meel, K.S.: Algorithmic fairness verification with graphical models. arXiv preprint arXiv:2109.09447 (2021)
[27]
Hardt, M., Price, E., Srebro, N.: Equality of opportunity in supervised learning. In: Advances in Neural Information Processing Systems, vol. 29 (2016)
[28]
Henzinger, T., Karimi, M., Kueffner, K., Mallik, K.: Runtime monitoring of dynamic fairness properties. In: Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pp. 604–614 (2023)
[29]
Henzinger TA, Karimi M, Kueffner K, and Mallik K Enea C and Lal A Monitoring algorithmic fairness Computer Aided Verification 2023 Cham Springer 358-382
[30]
Henzinger, T.A., Kueffner, K., Mallik, K.: Monitoring algorithmic fairness under partial observations. arXiv preprint arXiv:2308.00341 (2023)
[31]
Henzinger TA and Saraç NE Deshmukh J and Ničković D Monitorability under assumptions Runtime Verification 2020 Cham Springer 3-18
[32]
Henzinger, T.A., Saraç, N.E.: Quantitative and approximate monitoring. In: 2021 36th Annual ACM/IEEE Symposium on Logic in Computer Science (LICS), pp. 1–14. IEEE (2021)
[33]
Jerison, D.: General mixing time bounds for finite Markov chains via the absolute spectral gap. arXiv preprint arXiv:1310.8021 (2013)
[34]
John, P.G., Vijaykeerthy, D., Saha, D.: Verifying individual fairness in machine learning models. In: Conference on Uncertainty in Artificial Intelligence, pp. 749–758. PMLR (2020)
[35]
Kontorovich A and Raginsky M Carlen E, Madiman M, and Werner EM Concentration of measure without independence: a unified approach via the martingale method Convexity and Concentration 2017 New York Springer 183-210
[36]
Lahoti, P., Gummadi, K.P., Weikum, G.: iFair: learning individually fair data representations for algorithmic decision making. In: 2019 IEEE 35th International Conference on Data Engineering (ICDE), pp. 1334–1345. IEEE (2019)
[37]
Levin, D.A., Peres, Y.: Markov Chains and Mixing Times, vol. 107. American Mathematical Society (2017)
[38]
Liu, L.T., Dean, S., Rolf, E., Simchowitz, M., Hardt, M.: Delayed impact of fair machine learning. In: International Conference on Machine Learning, pp. 3150–3158. PMLR (2018)
[39]
Lum K and Isaac W To predict and serve? Significance 2016 13 5 14-19
[40]
Maler O and Nickovic D Lakhnech Y and Yovine S Monitoring temporal properties of continuous signals Formal Techniques, Modelling and Analysis of Timed and Fault-Tolerant Systems 2004 Heidelberg Springer 152-166
[41]
Meyer, A., Albarghouthi, A., D’Antoni, L.: Certifying robustness to programmable data bias in decision trees. In: Advances in Neural Information Processing Systems, vol. 34, 26276–26288 (2021)
[42]
Obermeyer Z, Powers B, Vogeli C, and Mullainathan S Dissecting racial bias in an algorithm used to manage the health of populations Science 2019 366 6464 447-453
[43]
Otop, J., Henzinger, T.A., Chatterjee, K.: Quantitative automata under probabilistic semantics. Logical Methods Comput. Sci. 15 (2019)
[44]
Paulin, D.: Concentration inequalities for Markov chains by Marton couplings and spectral methods (2015)
[45]
Ruchkin I, Sokolsky O, Weimer J, Hedaoo T, and Lee I Compositional probabilistic analysis of temporal properties over stochastic detectors IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 2020 39 11 3288-3299
[46]
Scheuerman, M.K., Paul, J.M., Brubaker, J.R.: How computers see gender: an evaluation of gender classification in commercial facial analysis services. Proc. ACM Hum.-Comput. Interact. 3(CSCW), 1–33 (2019)
[47]
Sharifi-Malvajerdi, S., Kearns, M., Roth, A.: Average individual fairness: algorithms, generalization and experiments. In: Advances in Neural Information Processing Systems, vol. 32 (2019)
[48]
Stoller SD et al. Khurshid S, Sen K, et al. Runtime verification with state estimation Runtime Verification 2012 Heidelberg Springer 193-207
[49]
Sun B, Sun J, Dai T, and Zhang L Huisman M, Păsăreanu C, and Zhan N Probabilistic verification of neural networks against group fairness Formal Methods 2021 Cham Springer 83-102
[50]
Waudby-Smith, I., Arbour, D., Sinha, R., Kennedy, E.H., Ramdas, A.: Time-uniform central limit theory, asymptotic confidence sequences, and anytime-valid causal inference. arXiv preprint arXiv:2103.06476 (2021)
[51]
Wexler J, Pushkarna M, Bolukbasi T, Wattenberg M, Viégas F, and Wilson J The what-if tool: interactive probing of machine learning models IEEE Trans. Vis. Comput. Graph. 2019 26 1 56-65
[52]
Younes HLS and Simmons RG Brinksma E and Larsen KG Probabilistic verification of discrete event systems using acceptance sampling Computer Aided Verification 2002 Heidelberg Springer 223-235
[53]
Zemel, R., Wu, Y., Swersky, K., Pitassi, T., Dwork, C.: Learning fair representations. In: International Conference on Machine Learning, pp. 325–333. PMLR (2013)
Recommendations
Runtime Monitoring of Dynamic Fairness Properties
FAccT '23: Proceedings of the 2023 ACM Conference on Fairness, Accountability, and TransparencyA machine-learned system that is fair in static decision-making tasks may have biased societal impacts in the long-run. This may happen when the system interacts with humans and feedback patterns emerge, reinforcing old biases in the system and creating ...
Comments
Information & Contributors
Information
Published In
Oct 2023
493 pages
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 03 October 2023
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 01 Jan 2025