Can Inequality Be Measured? Techniques for Assessing Algorithmic Bias

ABSTRACT This article asks whether inequality produced by artificial intelligence systems can be measured in a manner that is technically meaningful and legally useful. It argues that algorithmic bias is measurable only when three conditions are met: the protected or vulnerable groups are identified before testing, the statistical metric is selected by reference to the legal and factual context of use, and the result is interpreted through a governance process rather than treated as a self-executing answer. The analysis is situated in European Union law, especially Regulation (EU) 2024/1689 (Artificial Intelligence Act), and is methodologically grounded in the Spanish AI Supervisory Agency’s (AESIA) Guide on data and data governance, Annex C, and UNE-CEN/CLC ISO/IEC/TS 12791:2024. The article develops an operational framework for employment-related and biometric systems, using adverse impact, true positive rate differences, false negative rate differences and statistical robustness as examples of measurable thresholds. Empirical illustrations from automated recruitment and facial recognition show both the promise and the limits of measurement. The conclusion is deliberately cautious: metrics can reveal inequality, structure regulatory evidence and trigger mitigation duties, but they cannot replace the normative judgment required by equality law, data protection law and fundamental rights impact assessment. The article’s working hypothesis is that algorithmic inequality can be measured as evidence, but cannot be resolved through metrics alone for the purposes of legal compliance. Its practical contribution lies in proposing a rights-sensitive assessment model that integrates data provenance, disaggregated performance metrics, context-dependent acceptance thresholds, lifecycle governance controls, and residual risk documentation.

KEYWORDS Algorithmic bias - Non-discrimination - Artificial Intelligence Act (AI Act) - Fairness metrics - Fundamental rights impact assessment - High-risk AI systems - Bias mitigation - Data governance

TABLE OF CONTENTS 1. Introduction. – 2. Bias as a Lifecycle Risk. – 2.1. Why Bias Cannot Be Reduced to Bad Data. – 2.2. The Legal Object of Measurement. – 3. Techniques for Measuring Algorithmic Inequality. – 3.1. Disaggregated Performance as the Minimum Baseline. – 3.2. Group Fairness Metrics. – 3.3. Counterfactual and Proxy Analysis. – 4. A Methodological Framework for Acceptance Criteria. – 4.1. From Inherent Risk to Residual Risk. – 5. Empirical Illustrations. – 5.1. Automated Recruitment. – 5.2. Facial Recognition. – 5.3. Case-Law Summary: SyRI. – 6. Measurement Under EU Regulatory Governance. – 6.1. Provider Duties: Data, Documentation and Risk Management. – 6.2. Deployer Duties and Fundamental Rights Impact Assessment. – 6.2.1. From Legislative Silence to the Article 27 Duty. – 6.2.2. Subjective and Objective Scope of the Duty. – 6.2.3. Minimum Content: From Template to Reasoned File. – 6.2.4. Comparative Methodological Lessons. – 6.2.5. Avoiding Cosmetic Compliance. – 6.3. Special Categories of Data and the Paradox of Bias Testing. – 7. Treatment Measures and Standardisation. – 7.1. Implementation Challenges in the Public Sector. – 8. Conclusion.