A key element of child protection is to amplify the business sector’s participation as a partner in the process. This is particularly challenging in regard to the expanding mass of child sexual exploitation and abuse materials (CSAM) in a world of digitalisation, algorithms and artificial intelligence (AI).
A recent report backed by the UN concerning the impact of AI on child sexual exploitation notes that Cyber Tipline received some 36 million CSAM reports in 2023. There are the traditional forms of exploitation, such as photos of children exploited in prostitution, pornography and human trafficking, now coupled with more modern forms, such as suggestive texts used for grooming children and livestreaming.
Sextortion, namely blackmail threatening children with exposure of sexual materials involving them, is also a worrying phenomenon in the expansive crucible of online scams and deception.
The advent of AI complicates the scenario. It can “nudify” ordinary pictures of children and adults by converting them into sexualised images. This might be ordered by a text to create an abusive new image or by manipulating an older image to become sexualised.
There then arises the question: what are the costs and benefits for the sector in harnessing its cooperation more effectively?
The preferred entry point is to follow the UN’s Guiding Principles on Business and Human Rights, which underline the business sector’s duty to respect human rights, especially through due diligence measures, and to share with the state the duty to remedy the harm to target groups. The state itself is under the duty to protect human rights, such as by adopting good laws.
There are at least two approaches to the due diligence advocated. On the one hand, voluntary measures on the part of the business sector can be promoted. Voluntary action from the sector is based on self-regulation (“soft law”) such as business codes of conduct, community standards, terms of service (TOS) or contractual terms linked with TOS.
They can be coupled with oversight boards and personnel for content moderation, and technological tools and filters to block illegal and harmful content. The sector is encouraged to produce due diligence reports as a monitoring tool for harm prevention and mitigation, to use “notice and take down” requests to delete such content, and to make a range of remedies available.
On the other hand, the state or a regional organisation to which it belongs (such as the European Union) might seek to impose stricter measures by means of legislation (“hard law”) such as online safety and child protection laws, a digital services act or AI related-regulations, with mandatory reporting by the business sector to oversight mechanisms for the purpose of transparency, subject to hefty fines in the case of non-compliance. Australia’s online safety law, effective from 2022, follows this approach. There might also be a mix of the hard law and soft law approaches.
How expensive is it for the business sector to be involved in such measures? Disaggregation of the various measures shows that expenses vary and do not need to be high.
For example, including a provision on child protection in the contractual terms of service does not necessarily impose an additional cost. Paradoxically, the cost might be incurred by the consumer who clicks “agree” without reading the contract terms! This implies that contractual terms need to be more reader-friendly and child-sensitive.
How expensive is adopting codes of conduct or community standards with provisions on child protection, and how costly is the oversight board, as well as the content moderators who help screen and take down illegal and harmful content?
Interestingly, one major digital platform gave its board about US$150 million (4.9 billion baht) a couple of years ago to help oversee the flow of materials on the internet and moderate abusive content.
However, the resources of that platform are in the billions. As for an individual content moderator, there is also subcontracting to technicians in developing countries to save costs, and a key consideration is the psychological support these people may need to mitigate the impact of having to review a plethora of abusive materials.
With regard to requests to take down objectionable content as part of a “notice and take down” channel, it should cost nothing (or very little) to the platform or the complainant; the request can be from the user to the platform or from the platform to the person/group responsible for the content or loading it.
How costly are the technological tools, such as filters, for identifying and eliminating these materials?
Some are free under open-source licences, while others can be costly. They range from filters to identify old abusive material, namely “hash matching”, to language processing tools to identify sextortion. A tool for assessing the possibility of AI producing offensive material is known as “red teaming”. At the same time, there is also a tool for labelling or watermarking AI-generated material for transparency, offered free to some developers.
As for due diligence reports, the cost of preparation by experts ranges from several thousand dollars for small platforms to substantial sums for mega-platforms. However, failure to submit such a report under hard law systems may result in substantial fines. The bottom line is that there is a need to integrate child protection measures into digital platforms and operations through a variety of means.
This is not necessarily expensive, as some facilities are free, e.g., various filters and technological tools available as open source. Bigger operations may need to meet higher standards, such as mandatory reporting requirements. Small operations may need assistance and incentives that the larger operations can offer, such as peer cooperation.
Where soft law (such as community standards) does not work or has a poor impact, a hard law (such as an online safety law, with mandatory reporting and related remedies) may be required. Multi-stakeholder action, including the business sector, is thus essential for child protection, with age-appropriate and gender-sensitivity.
Intrinsically, it makes good business sense for the sector to integrate child protection into its platforms because it is the right thing to do, builds responsible consumerism, and prevents harm that could lead to exponential accountability.