The EU built a system called CounterR that essentially performs pre-crime thought surveillance. The TLDR is that an AI company, with direct input from half a dozen European police forces, built a tool that scrapes social media, forums, and other sources to assign citizens a score based on what they think as opposed to what they’ve actually done. The EC also has not released details of the project..
The report itself acknowledges that this sort of automated system “can trigger new fundamental rights risks that affect rights different than the protection of personal data and privacy.”
The police were active co-developers, sitting in meetings to define the criteria and feeding real, anonymized data from their investigations to train the LLM. So now you have a feedback loop where police define the threat, the LLM learns it, and the police validate the results, with zero external oversight.
And of course, it’s all shrouded in secrecy. The whole thing is confidential, the source code is proprietary so even partners can’t audit it, and the ethics board is made up of the same people building the thing. There’s no clear requirement to track false positives, so you could be flagged as a potential radical and never know why.
The cherry on top? The core technology, developed with public funds, was recently acquired by a private company, Logically, who can now sell this dystopian scoring system to whoever they want.
The citizens of the EU literally paid to build our own panopticon. The whole project is about normalizing the idea that the state gets to algorithmically monitor and judge your political beliefs before you ever commit a crime.
I can’t help but think that sooner or later, someone is gonna try combining AI with the massive amount of data collected on individuals these days to effectively put us in a cheap knockoff of Minority Report.
Or maybe the tinfoil is finally attaching itself to my scalp…
The incumbents in EU are very scared as politicians from outside the traditional political families are getting popular votes. And instead of looking into to mirror as to why that is happening, they blame “the internet” and go authoritarian.
Another example: EU parliament also accepted a last minute amendment, mandating age verification for pornographic (whatever that is) content online, punishable with up to one year prison sentence.
This was a last minute addition to a directive concerning CSAM. Because adults accessing porn need to be de-anonymised to avoid child exploitation?
The EU apparat is self-serving, enjoying their private internal turmoil and drama. Annoyed by those pesky voters and their stupid wants and rights. If only there was software we could use to get them to shut up?!
There was a book written about this once, almost a cautionary tale.
The concept of a thoughtcrime is the result of a totalitarian society, but it’s also a concept that is born out of fear of what we’re allowed to say.
So what happens when you ask the robot if it’s penis is metal?
Sure, religious conservatives always have a hand in making up “who’s evil and should be scored that way”. So, you’re fucked if you’re not their religion, or particularly if you’re an atheist.
Like every palestinian protester, you’d be considered the same as a WWII nazi for antisemetic beliefs. Because you don’t want jewish people genociding muslim people, or vice-versa.
That’s right. You get put on a list as a radical, because you DON’T support genocide.
What’s the source for this programme scoring individuals ? Per the EU project description, the target is specifically communities at large rather than individuals.