Police car on a road
Image by Adrian Tworuschka, Pixabay.

Artificial intelligence as a basis for police searches

Does the German state use data from the United States that has been evaluated by artificial intelligence as the basis for police searches? The Centre for Human | Data | Society at the University of Konstanz warns that digital processes are increasingly precipitating legal regulations. As a result, algorithms are gaining quasi-legal power.

It sounds like a futuristic idea: Artificial intelligence (AI) analyzes data from millions of reported crimes, decides which could be relevant for criminal prosecution, and triggers real-life police searches of people's homes as a result. Yet, there are good reasons to conclude that this is already happening in Germany. The US American platform "CyberTipline" enacted by the non-profit National Center for Missing & Exploited Children (NCMEC) allegedly sends German authorities data on reported child sexual abuse material (CSAM) that has been analyzed using artificial intelligence. In Germany, this data is being used in a relatively unfiltered way as a basis for police searches, says prosecutor Simon Pschorr who works at the University of Konstanz. The university's Centre for Human | Data | Society (CHDS) sees a transparency problem when AI-processed data is used in the context of criminal investigations and warns against giving an algorithm quasi-legal power.

32 million tip-offs, 350 employees and presumably one AI
The NCMEC is a private, non-profit child protection organization with some responsibilities delegated to it by the state. The organization set up the online platform "CyberTipline" for people to report suspected CSAM. US American internet service providers are even legally obligated to do so.

In 2022 alone, the platform received more than 32 million tip-offs – about 87,600 per day on average. A team of around 350 staff, including 30 analysts, is tasked with sifting through this mountain of reported crimes. Yet, the data is processed very quickly, at times even within 28 seconds of being reported as indicated in a local court decision by Amtsgericht Reutlingen, and the report is passed on to German authorities.

"It is not physically possible for people to look at, evaluate and pass along this information within seconds. There had to have been an automated system involved", concludes prosecutor Simon Pschorr, a researcher at the Centre for Human | Data | Society. Pschorr suspects that artificial intelligence is used to process the data. Additional indications that this is taking place are the identical formatting of reports provided by CyberTipline as well as the well-known cooperation between NCMEC (the website owner) and the US American software company Palantir that develops AI solutions for processing large amounts of data.

Legal problems and potential solutions
In the fight against CSAM, the NCMEC and its CyberTipline platform are considered exemplary. According to Simon Pschorr, in well over half of the reported cases, police searches find CSAM at the respective addresses. He thus does not call the usefulness of this institution into question, but he does voice fundamental concerns about the legal justification for obtaining evidence in this way.

  • Data transparency: It is unclear how the AI system used by CyberTipline reached its conclusions. There is no documentation of the steps and evidence employed by the algorithm. This means that the use of this data for prosecution in criminal proceedings is on very thin ice. In fact, very few charges are brought directly on the basis of the data. In most cases, the data reported by CyberTipline lays the foundation for police searches. The indictment is then based on evidence found during the searches, not on the reported data.
  • Circumvention of German law: According to a decision by the Federal Constitutional Court in 2023, automatic data processing systems in Germany may not be used to process police data without a specific legal basis. German authorities circumvent this regulation by using data that was automatically processed abroad to open the door for police searches. Simon Pschorr argues, "We cannot allow authorities to shop around for the least restrictive national legislation. The CyberTipline serves a good purpose – but the end doesn't justify the means".
  • Lack of a state institution: The CyberTipline is owned by a private organization, not a state institution. In this case, a private organization is entrusted with the state's responsibilities to document and combat crime, and there is no judicial control.

In order to resolve the problem, Simon Pschorr proposes establishing a European equivalent of the CyberTipline that a) conducts its work in line with European law, b) ensures data transparency and c) has a state (or European) owner. He says this would effectively protect basic rights and avoid "prosecution at any cost".

When the algorithm becomes law
The Centre for Human | Data | Society observes that, as datafication increases, there is a growing tendency to give algorithms quasi-legal power. The CyberTipline is just one example of this. "Computer code creates law – the algorithm becomes a law onto itself", warns Liane Wörner, director of the Centre for Human | Data | Society. "We observe that there is regulatory chaos in the digital space, and responsibilities are often left undefined. Programming is used to establish regulations that have quasi-legislative power". All too often, the programmable possibilities define legal practice in the digital sector – instead of real legal regulations.

About the Centre for Human | Data | Society
In autumn of 2022, the Centre for Human | Data | Society (CHDS) was founded at the University of Konstanz. At the centre, researchers investigate the processes of digitalization and datafication in our (data) society, putting people centre stage: What interactions are there between humans and the data society? What kind of data society do we really want, and how should it be designed? To answer these questions, the CHDS team employs a transdisciplinary approach analyzing technical, legal, political, psychological, media-cultural, historical, economic and social aspects of the data society. Further information: www.uni-konstanz.de/centre-for-human-data-society/  

Key facts:

[Translate to Englisch:] Staatsanwalt Simon Pschorr, abgeordneter Praktiker an der Universität Konstanz. Copyright: Simon Pschorr
[Translate to Englisch:] Prof. Dr. Liane Wörner, Professorin für Strafrecht, Strafprozessrecht, Strafrechtsvergleichung, Medizinstrafrecht und Rechtstheorie an der Universität Konstanz und Direktorin des Centre for Human | Data | Society. Copyright: Universität Konstanz, Inka Reiter