Polish Researcher Wins €2.5 Million ERC Grant to Study AI Bias in Criminal Justice

University of Wrocław has received its first-ever grant from the European Research Council (ERC Advanced Grant), and its recipient is immediately tackling one of the most sensitive problems in AI and law: systematic algorithmic bias against women and minorities in criminal proceedings.

Kremens heads the Digital Justice Center at the Faculty of Law, Administration and Economics at University of Wrocław (UWr), which has for years studied the impact of new technologies on criminal proceedings. The ERC Advanced Grant, awarded to researchers with an established track record and ambitious research projects, will let her lead a team for five years focused on a single question: what happens to the principles of criminal procedure when they collide with digital tools built on data loaded with bias.

Where AI bias comes from

The project's starting point is the claim that criminal law, despite its appearance of neutrality, was built on male experience and only later declared universal. Kremens applies this argument to technologies used today by law enforcement, pointing to a specific mechanism: facial recognition systems trained mainly on images of men and white women perform worst precisely at identifying Black women.

Law was created based on male experience, and then declared universal - Prof. Karolina Kremens, University of Wrocław

This is not an abstract academic problem. Misidentification by a facial recognition system means, in practice, false suspicion, detentions, or evidentiary motions built on flawed premises. The researcher argues that since these kinds of tools are already reaching prosecutors and police, and criminal procedure has no mechanisms to correct for this bias, the risk of error falls systematically on groups that are already in a weaker evidentiary position.

Asymmetry between prosecution and defense

The second thread concerns the procedural inequality introduced by the digitization of criminal proceedings. Kremens notes that the digital revolution is already undermining fundamental principles of criminal procedure, such as immediacy and equality of arms: prosecutors have access to advanced AI tools for analyzing digital evidence that a defendant mounting a defense often cannot access or verify.

In practice, this means the side with greater technological and financial resources gains an advantage independent of the merits of the case. The crimPROfem project aims to examine how large a role this asymmetry plays and whether it can be reduced through regulation before AI tools become a permanent fixture in the standard toolkit of courts and prosecutors across Europe.

Method: rewriting verdicts

As part of the project, Kremens's team plans to literally rewrite selected criminal verdicts from several countries, applying tools of feminist legal theory to them to test whether the conclusions or reasoning would change if the evidence and digital communications were assessed from that perspective. This method is known from similar legal projects abroad but is rarely applied to AI-based forensic technologies.

The researcher stresses that reforms benefiting one group of women can harm another. As an example, she cites mandatory arrest of domestic violence perpetrators: a policy introduced to protect victims can, in practice, push families already in a difficult financial situation into poverty if the sole breadwinner is automatically jailed without regard to context. The project is therefore meant to take an intersectional perspective: race, class, sexual orientation, education and financial situation, not just gender.

What it means for Poland

For Polish science, this is an event in itself: University of Wrocław had never won an ERC grant before, and in this year's Advanced Grants competition it was the only Polish success story. The prestige of the ERC Advanced Grant lies precisely in the fact that it is awarded to a small number of researchers with recognized track records, to carry out risky but potentially breakthrough research projects, funded on a large scale and over a long period.

For Polish lawyers and developers of AI systems used in the justice system, the crimPROfem project could, over the next five years, deliver concrete recommendations on how to design criminal procedures resistant to algorithmic bias before similar tools become firmly established in Polish courts and prosecutors' offices. The study's findings could also feed into the discussion on standards for AI use in law enforcement currently underway at the EU level as part of the implementation of the AI Act.