Facebook Delivers AI To Detect Suicidal Posts

This is software to save lives. Facebook’s new “proactive detection” artificial intelligence technology will scan all posts for patterns of suicidal thoughts, and when necessary send mental health resources to the user at risk or their friends, or contact local first-responders. 

By using AI to flag worrisome posts to human moderators instead of waiting for user reports, Facebook can decrease how long it takes to send help.

Facebook previously tested using AI to detect troubling posts and more prominently surface suicide reporting options to friends in the US. 

Now Facebook is will scour all types of content around the world with this AI, except in the European Union, where General Data Protection Regulation privacy laws on profiling users based on sensitive information complicate the use of this tech.
Facebook also will use AI to prioritise particularly risky or urgent user reports so they’re more quickly addressed by moderators, and tools to instantly surface local language resources and first-responder contact info. It’s also dedicating more moderators to suicide prevention, training them to deal with the cases 24/7, and now has 80 local partners like Save.org, National Suicide Prevention Lifeline and Forefront from which to provide resources to at-risk users and their networks.
“This is about shaving off minutes at every single step of the process, especially in Facebook Live,” says VP of product management Guy Rosen. Over the past month of testing, Facebook has initiated more than 100 “wellness checks” with first-responders visiting affected users. “There have been cases where the first-responder has arrived and the person is still broadcasting.”

The idea of Facebook proactively scanning the content of people’s posts could trigger some dystopian fears about how else the technology could be applied. Facebook didn’t have answers about how it would avoid scanning for political dissent or petty crime, with Rosen merely saying “we have an opportunity to help here so we’re going to invest in that.” 

There are certainly massive beneficial aspects about the technology, but it’s another space where we have little choice but to hope Facebook doesn’t go too far.

Facebook CEO Mark Zuckerberg praised the product update in a post today, writing that “In the future, AI will be able to understand more of the subtle nuances of language, and will be able to identify different issues beyond suicide as well, including quickly spotting more kinds of bullying and hate.”

According to TechCrunch there is no way for Facebook users to opt out of having their posts scanned. A Facebook spokesperson noted that the feature is designed to enhance user safety and that support resources offered by Facebook can be quickly dismissed if a user doesn’t want to see them.

Facebook trained the AI by finding patterns in the words and imagery used in posts that have been manually reported for suicide risk in the past. It also looks for comments like “are you OK?” and “Do you need help?”
“We’ve talked to mental health experts, and one of the best ways to help prevent suicide is for people in need to hear from friends or family that care about them,” Rosen says. “This puts Facebook in a really unique position. We can help connect people who are in distress connect to friends and to organizations that can help them.”
How suicide reporting works on Facebook now

Through the combination of AI, human moderators and crowdsourced reports, Facebook could try to prevent tragedies like when a father killed himself on Facebook Live last month. 

Live broadcasts in particular have the power to wrongly glorify suicide, hence the necessary new precautions, and also to affect a large audience, as everyone sees the content simultaneously unlike recorded Facebook videos that can be flagged and brought down before they’re viewed by many people.

Now, if someone is expressing thoughts of suicide in any type of Facebook post, Facebook’s AI will both proactively detect it and flag it to prevention-trained human moderators, and make reporting options for viewers more accessible.
When a report comes in, Facebook’s tech can highlight the part of the post or video that matches suicide-risk patterns or that’s receiving concerned comments. That avoids moderators having to skim through a whole video themselves. 

AI prioritises users reports as more urgent than other types of content-policy violations, like depicting violence or nudity. Facebook says that these accelerated reports get escalated to local authorities twice as fast as un-accelerated reports.
Facebook’s tools then bring up local language resources from its partners, including telephone hotlines for suicide prevention and nearby authorities. 

The moderator can then contact the responders and try to send them to the at-risk user’s location, surface the mental health resources to the at-risk user themselves or send them to friends who can talk to the user. “One of our goals is to ensure that our team can respond worldwide in any language we support,” says Rosen.

Back in February, Facebook CEO Mark Zuckerberg wrote that “There have been terribly tragic events, like suicides, some live streamed, that perhaps could have been prevented if someone had realised what was happening and reported them sooner.   Artificial intelligence can help provide a better approach.”

With more than 2 billion users, it’s good to see Facebook stepping up here. Not only has Facebook created a way for users to get in touch with and care for each other. It’s also unfortunately created an unmediated real-time distribution channel in Facebook Live that can appeal to people who want an audience for violence they inflict on themselves or others.

Creating a ubiquitous global communication utility comes with responsibilities beyond those of most tech companies, which Facebook seems to be coming to terms with.

Techcrunch:

You Might Also Read:

Social Media - 'Jargon-Busted':

Give Children More Control Of Data Privacy:


 

 

« Staff Training Is Important But Does Not Reduce Cyber Risk
China Intends To Be An AI Superpower »

CyberSecurity Jobsite
Perimeter 81

Directory of Suppliers

Resecurity, Inc.

Resecurity, Inc.

Resecurity is a cybersecurity company that delivers a unified platform for endpoint protection, risk management, and cyber threat intelligence.

Perimeter 81 / How to Select the Right ZTNA Solution

Perimeter 81 / How to Select the Right ZTNA Solution

Gartner insights into How to Select the Right ZTNA offering. Download this FREE report for a limited time only.

CSI Consulting Services

CSI Consulting Services

Get Advice From The Experts: * Training * Penetration Testing * Data Governance * GDPR Compliance. Connecting you to the best in the business.

ManageEngine

ManageEngine

As the IT management division of Zoho Corporation, ManageEngine prioritizes flexible solutions that work for all businesses, regardless of size or budget.

Jooble

Jooble

Jooble is a job search aggregator operating in 71 countries worldwide. We simplify the job search process by displaying active job ads from major job boards and career sites across the internet.

Stott & May

Stott & May

Stott & May is a specialist cyber security recruitment agency.

International School of IT Security (ISITS)

International School of IT Security (ISITS)

The International School of IT Security (ISITS) is a leading provider of professional training in the field of IT Security.

CISPA Helmholtz Center for Information Security

CISPA Helmholtz Center for Information Security

The CISPA Helmholtz Center for Information Security is a German national Big Science Institution within the Helmholtz Association. Our research encompasses all aspects of Information Security.

National Institute of Information and Communications Technology (NICT)

National Institute of Information and Communications Technology (NICT)

NICT is Japan’s sole National Research and Development Agency specializing in the field of information and communications technology.

Logsign

Logsign

Logsign is a Security Orchestration, Automation and Response (SOAR) platform with next-gen Security Information and Event Management (SIEM) solution.

Cingo Solutions

Cingo Solutions

Cingo Solutions is a Managed Detection & Response company providing specialized data security services.

WWPass

WWPass

WWPass is a global cybersecurity company that provides password-less authentication and client-side encryption technology.

New Enterprise Associates (NEA)

New Enterprise Associates (NEA)

As one of the world’s largest and most active venture capital firms, NEA has developed deep domain expertise and insight into our industries of focus - technology and healthcare.

Police Digital Security Centre (PDSC)

Police Digital Security Centre (PDSC)

PDSC is a not-for-profit organisation, owned by the police, that works across the UK in partnership with industry, government, academia and law enforcement.

BlackCloak

BlackCloak

BlackCloak provides Concierge Cyber Security for high-net-worth individuals and corporate executives to protect them from cybercrime, reputational risks, hacking and identity theft.

Maven Security Consulting

Maven Security Consulting

Maven Security Consulting helps companies secure their information assets and digital infrastructure by providing a wide range of customized consulting and training services.

Take Five

Take Five

Take Five is a national campaign offering straight-forward, impartial advice that helps prevent email, phone-based and online fraud – particularly where criminals impersonate trusted organisations.

Flix11

Flix11

Flix11 is a Cyber Security & ICT Solutions focused company. We provide a range of products and services in Cyber Security, Internet of Things (IoT) and infrastructure solutions.

CySecK

CySecK

CySecK is a Centre of Excellence in Cybersecurity formed in 2017 by the Government of Karnataka, as part of the Technology Innovation Strategy.

Locuz

Locuz

At Locuz, we’ve made it our mission to help businesses like yours create an actionable digital strategy.

Safe Data Storage

Safe Data Storage

Safe Data Storage offer a fully managed, professional, secure UK-based online backup service to businesses, education and charities.