IN an attempt to help parents guide their children away from self-harm, Meta AI rolled out on Instagram a supervision tool for parents to help them become aware of distressing and sensitive conversations made by their children regarding suicide or self-harm.
Announced on Meta newsroom, a new and dedicated AI system, with the help of parents and experts, was developed to identify disturbing conversations that would warrant an alert to parents.
In this new AI system, if the children talked about having thoughts of self-harm or suicide, they would be directed to crisis helplines and would be encouraged to talk to a trusted adult. Moreover, it will send an alert to parents to inform them about these conversations so they can provide guidance to the child.
Meta, meanwhile, emphasized that chats flagged by the AI will be manually reviewed before the AI sends an alert. Additionally, these alerts will come with a shared resource from experts so parents would also have a guide on how to handle this problem tactfully.
‘While I believe that teens have a right to privacy, I also believe parents need to be informed if their teen may be at risk of hurting themselves. That’s why I advocated for this approach and support Meta’s decision to notify parents when, after careful review, it determines that a conversation with Meta AI contains indications of possible suicide or self-harm that warrant an alert,’ said Larry Magid, CEO and co-founder of ConnectSafely.
‘I appreciate how Meta struck the right balance; protecting teen privacy while ensuring parents have the information they need to support their teen,’ he added.
Experts assisted Meta by reviewing conversational prompts that Meta AI will flag.
‘I was struck by the rigor of Meta’s clinical review process. It examined not only immediate responses to suicide and self-harm concerns, but also the broader conversational context, appropriate follow-up, and the varying levels of risk that can exist even within high-risk situations. This kind of expert-informed, scenario-based refinement is essential to making AI experiences safer for teens,’ said USA-licensed psychologist Dr. Ji-yeon Lee.
This initiative is currently live for parents who use Instagram Parental Supervision in the United States, United Kingdom, Australia, and Canada, with the global rollout planned by the end of the year.