School leaders to debate criminalising non-consensual deepfake content, says NAHT

In a recent report, Dame Rachel de Souza, the children’s commissioner for England, warned that generative AI apps that enable the creation of sexual deepfake images should be banned.

The report analyses the threat of nudification technology, assessing its use online and the impact on children. Key concerns included the high risk of harm to children, the targeting of girls by AI, the ease of access to these tools, and the preventative steps that children are taking to keep themselves safe.

Dame Rachel de Souza said: “In our lifetime, we have seen the rise and power of Artificial Intelligence (AI) – once the stuff of science fiction – to shape the way we learn, connect and experience the world. It has enormous potential to enhance our lives, but in the wrong hands it also brings alarming risks to children’s safety online. 

“Children have told me they are frightened by the very idea of this technology even being available, let alone used. They fear that anyone – a stranger, a classmate, or even a friend – could use a smartphone as a way of manipulating them by creating a naked image using these bespoke apps.

“Girls have told me they now actively avoid posting images or engaging online to reduce the risk of being targeted by this technology – we cannot sit back and allow these bespoke AI apps to have such a dangerous hold over children’s lives. 

“The online world is revolutionary and quickly evolving, but there is no positive reason for these particular apps to exist. They have no place in our society. Tools using deepfake technology to create naked images of children should not be legal and I’m calling on the government to take decisive action to ban them, instead of allowing them to go unchecked with extreme real-world consequences.” 

Paul Whiteman, general secretary of school leaders’ union NAHT, commented: “School leaders share the children’s commissioner’s concerns over this technology, both in its use against children and against school staff. Our members will be debating this subject at our conference next week, calling for the creation and distribution of non-consensual deepfake content to be criminalised.

“This is an area that urgently needs to be reviewed as the technology risks outpacing the law and education around it.”

The children’s commissioner is calling for urgent action including specific legal responsibilities for developers of the AI apps, effective systems to remove sexually explicit deepfake images of children from the internet, the recognition of deepfake sexual abuse as a form of violence against women and girls, and the outright banning of such nudification apps.

School leaders at NAHT will debate and vote on the motion on Saturday, 3 May, in Harrogate at the union’s annual conference.