ARTICLE AD BOX
The Children’s Commissioner is calling for generative AI apps to be banned, saying they are being used to create sexually explicit images of children.
Dame Rachel de Souza wants the Government to ban all apps that allow for ‘nudification’ (where photos of real people are edited by AI to make them appear naked).
She also wants a ban on any apps that can be used to create explicit deepfake images of young people.
Generative AI tools are already widely available and often free to use, Dame Rachel warned.
It is illegal to create or share a sexually explicit image of a child, but the technology enabling it is legal and can be easily found online, she said.
Dame Rachel has published a new report on the subject, in which she says children, and in particular girls, were changing their online behaviour so as not to become victims of nudification apps.
“In our lifetime, we have seen the rise and power of artificial intelligence – once the stuff of science fiction – to shape the way we learn, connect and experience the world.
“It has enormous potential to enhance our lives, but in the wrong hands it also brings alarming risks to children’s safety online,” Dame Rachel said.
“Children have told me they are frightened by the very idea of this technology even being available, let alone used.
“They fear that anyone – a stranger, a classmate, or even a friend – could use a smartphone as a way of manipulating them by creating a naked image using these bespoke apps.
“Girls have told me they now actively avoid posting images or engaging online to reduce the risk of being targeted by this technology. We cannot sit back and allow these bespoke AI apps to have such a dangerous hold over children’s lives.
“The online world is revolutionary and quickly evolving, but there is no positive reason for these particular apps to exist. They have no place in our society.
“Tools using deepfake technology to create naked images of children should not be legal and I’m calling on the Government to take decisive action to ban them, instead of allowing them to go unchecked with extreme real-world consequences.”
As well as the ban on bespoke nudification apps, the report calls for creating specific legal responsibilities for developers of generative AI tools to cut the risks children could face, and to establish new systems to help remove sexually explicit deepfakes of children from the internet.
Dame Rachel also calls for deepfake sexual abuse to be recognised as a form of violence against women and girls, and that it be taken seriously in both law and policy.
Paul Whiteman, general secretary of school leaders’ union NAHT, said its members shared her concerns about the technology, both in its use against children and against school staff.
“Our members will be debating this subject at our conference next week, calling for the creation and distribution of non-consensual deepfake content to be criminalised,” he said.
“This is an area that urgently needs to be reviewed as the technology risks outpacing the law and education around it.”
In response, a Government spokesperson said: “Creating, possessing or distributing child sexual abuse material, including AI-generated images, is abhorrent and illegal.
“Under the Online Safety Act platforms of all sizes now have to remove this kind of content, or they could face significant fines.
“The UK is the first country in the world to introduce further AI child sexual abuse offences – making it illegal to possess, create or distribute AI tools designed to generate heinous child sex abuse material.”
English (US) ·