The EU Horizon project brings together an interdisciplinary consortium of nine partner institutions to develop a deep understanding of the use of AI in the employment sector and to detect & mitigate unfairness in AI-driven recruitment tools
Situation
Artificial Intelligence (AI) is increasingly used in the employment sector. A recent Sage study found that 24% of companies use AI for recruitment purposes. This often involves Natural Language Processing (NLP) based AI-models to analyze text created by a job candidate. High profile cases have shown that such systems can reproduce social prejudices and unfairly discriminate against underrepresented minorities. This form of algorithmic bias is exacerbated by the fact that AI decision-making processes usually occur in a black-box, opaque even to the engineers who designed them. This results in systems capable of rendering unjust and unjustified decisions with low accountability, decisions that are often not subject to appeal on behalf of adversely affected human stakeholders. In practice, machine learning (ML) and NLP-based applications typically consist of off-the-shelf large-language models (LLMs) such as BERT, GPT models etc., which are then fine-tuned on a task-specific dataset; for example, an archive of job applications labeled according to whether the corresponding candidate was successful. Both the general language models employed, and the task-specific training data are potential sources of bias.
Objective
Artificial Intelligence (AI) is increasingly used in the employment sector to manage and control individual workers. One type of AI is Natural Language Processing (NLP) based tools that can analyze text to make inferences or decisions. A recent Sage study found that 24% of companies used AI for hiring purposes. In an employment context, this can involve analyzing text created by an employee or recruitment candidate in order to assist management in deciding to invite a candidate for an interview, to training and employee engagement, or to monitor for infractions that could lead to disciplinary proceedings. However, the models that NLP-based systems are based on are biased. Additionally, it has been shown that bias in an underlying AI model is reproduced in applications based on that model). This can lead to biased decisions that run contrary to the goals of the European Pillar of Social Rights in relationship to work and employment, specifically Pillar 2 (Gender Equality), Pillar 3 (Equal Opportunity), Pillar 5 (Secure and Adaptable Employment) and the United Nations’ (UN) Sustainable Development Goals (SDGs), specifically SDG 5 (Gender Equality), SDG 8 (Decent Work and Economic Growth). It is therefore necessary to identify and mitigate biases that occur in applications used in a Human Resources Management (HRM) context. Addressing such concerns in an employment context is especially relevant, as most existing European studies on employment discrimination have indeed found that discrimination exists, both when considering individual diversity criteria and multiple criteria in intersectional analyses. In order to investigate and mitigate these biases, we apply this “BIAS”-project, for mitigating diversity biases of AI in the labor market. The chief technical objective of BIAS is the development of a proof-of-concept for an innovative technology based on Natural Language Processing (NLP) and Case Based Reasoning (CBR) for use in an HR recruitment use case.
Last updated:10.03.2023
