When Fairness Metrics Conflict: A Cross-Domain Analysis of Algorithmic Bias and the Regulatory Specification Gap
##plugins.themes.bootstrap3.article.main##
Abstract
Algorithmic systems now govern consequential decisions in criminal justice, healthcare, and employment, yet persistent evidence demonstrates that these systems reproduce and amplify the very inequities they were designed to transcend. This paper advances the Fairness Metric Misalignment Framework (FMMF) as a structured analytical lens for understanding why algorithmic accountability failures persist even when individual actors comply with their respective obligations. Drawing on published audit studies, peer-reviewed empirical research, and regulatory documents across three high-stakes domains, the paper demonstrates that the root cause of accountability failure is not technical malfunction but metric selection incoherence: different stakeholders implicitly adopt incompatible fairness definitions, audit studies measure incommensurable outcomes, and regulatory frameworks mandate compliance with standards whose mathematical preconditions are mutually exclusive under real-world conditions. The FMMF organizes this diagnosis around three components: a metric taxonomy mapping the normative assumptions embedded in five major fairness definitions; a cross-domain mapping revealing systematic misalignment between the fairness metrics used in audit studies, system design objectives, and regulatory mandates; and a regulatory translation gap analysis demonstrating that major instruments including the European Union Artificial Intelligence Act (2024), the United States Equal Employment Opportunity Commission Technical Assistance (2023), and New York City Local Law 144 (2021) deploy fairness language without specifying the mathematical criteria that would render compliance verifiable. The paper concludes that adequate algorithmic governance requires regulators to specify fairness metrics explicitly, acknowledge the trade-offs their chosen metrics foreclose, and establish cross-domain coordination mechanisms to prevent governance fragmentation that insulates discriminatory systems from accountability. These findings carry direct implications for policymakers, system auditors, and institutional deployers navigating the emerging regulatory landscape for high-risk artificial intelligence.
Publish Your Research in This Journal
Open Journal of Interdisciplinary Studies welcomes original research submissions on related topics.
- Double-blind peer review
- First decision in 3–4 weeks
- DOI assigned on publication
- Open Access — free to read
##plugins.themes.bootstrap3.article.details##
References
Ahmad, A., Vallès, Y., & Idaghdour, Y. (2026). Bias in AI systems: Integrating formal and socio-technical approaches. Frontiers in Big Data, 8. https://doi.org/10.3389/fdata.2025.1686452
Angwin, J., Larson, J., Mattu, S., & Kirchner, L. (2016, May 23). Machine bias: There’s software used across the country to predict future criminals. And it’s biased against Blacks. ProPublica. https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
Chouldechova, A. (2017). Fair Prediction with Disparate Impact: A Study of Bias in Recidivism Prediction Instruments. Big Data, 5(2), 153–163. https://doi.org/10.1089/big.2016.0047
Dastin, J. (2018, October 11). Insight—Amazon scraps secret AI recruiting tool that showed bias against women. Reuters. https://www.reuters.com/article/world/insight-amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK0AG/
Dieterich, W., Mendoza, C., & Brennan, T. (2016). COMPAS risk scales: Demonstrating accuracy equity and predictive parity. Northpointe Inc. https://perma.cc/RVC7-SE8R
Dressel, J., & Farid, H. (2018). The accuracy, fairness, and limits of predicting recidivism. Science Advances, 4(1), eaao5580. https://doi.org/10.1126/sciadv.aao5580
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012). Fairness through awareness. Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, 214–226. https://doi.org/10.1145/2090236.2090255
EPIC. (2019). Complaint and Request for Investigation, Injunction, and Other Relief. The Electronic Privacy Information Center (EPIC). https://epic.org/wp-content/uploads/privacy/ftc/hirevue/EPIC_FTC_HireVue_Complaint.pdf
Equal Employment Opportunity Commission. (2023, May 18). Questions and Answers to Clarify and Provide a Common Interpretation of the Uniform Guidelines on Employee Selection Procedures. U.S. Equal Employment Opportunity Commission. https://www.eeoc.gov/laws/guidance/questions-and-answers-clarify-and-provide-common-interpretation-uniform-guidelines
European Parliament and Council. (2024). Regulation (EU) 2024/1689 of the European Parliament and of the Council of 13 June 2024 laying down harmonised rules on artificial intelligence and amending Regulations (EC) No 300/2008, (EU) No 167/2013, (EU) No 168/2013, (EU) 2018/858, (EU) 2018/1139 and (EU) 2019/2144 and Directives 2014/90/EU, (EU) 2016/797 and (EU) 2020/1828 (Artificial Intelligence Act) (Text with EEA relevance). European Parliament and Council of the European Union. http://data.europa.eu/eli/reg/2024/1689/oj
Hardt, M., Price, E., & Srebro, N. (2016). Equality of Opportunity in Supervised Learning. Advances in Neural Information Processing Systems, 29. https://proceedings.neurips.cc/paper_files/paper/2016/hash/6a9659feb1216f14f7384ba499518b38-Abstract.html
Harwell, D. (2019, November 6). Rights group files federal complaint against AI-hiring firm HireVue, citing ‘unfair and deceptive’ practices. The Washington Post. https://www.washingtonpost.com/technology/2019/11/06/prominent-rights-group-files-federal-complaint-against-ai-hiring-firm-hirevue-citing-unfair-deceptive-practices/
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2016). Inherent Trade-Offs in the Fair Determination of Risk Scores (arXiv:1609.05807). arXiv. https://doi.org/10.48550/arXiv.1609.05807
Kusner, M. J., Loftus, J., Russell, C., & Silva, R. (2017). Counterfactual Fairness. Advances in Neural Information Processing Systems, 30. https://proceedings.neurips.cc/paper/2017/hash/a486cd07e4ac3d270571622f4f316ec5-Abstract.html
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019). Dissecting racial bias in an algorithm used to manage the health of populations. Science, 366(6464), 447–453. https://doi.org/10.1126/science.aax2342
Office of the New York State Comptroller. (2025, December 2). Enforcement of Local Law 144 – Automated Employment Decision Tools. New York State Office of the Comptroller. https://www.osc.ny.gov/state-agencies/audits/2025/12/02/enforcement-local-law-144-automated-employment-decision-tools
Raghavan, M., Barocas, S., Kleinberg, J., & Levy, K. (2020). Mitigating bias in algorithmic hiring: Evaluating claims and practices. Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 469–481. https://doi.org/10.1145/3351095.3372828
##plugins.themes.bootstrap3.article.sidebar##
Article metrics
- Abstract Views 295
- PDF Downloads 0
- Total Engagements 295
How to Cite
How to Cite
This work is licensed under a Creative Commons Attribution 4.0 International License.