Social Media Companies Scan For Potential Terrorists

Following the politically motivated shooting in Pittsburgh and the mailing of pipe bombs to political officials and journalists across the country, public outcry has risen against social-media companies. 

Suspected pipe-bomber Cesar Sayoc and Pittsburgh shooting suspect Robert Bowers used various platforms to post content indicating their potential for ideological violence. 

Some have asked why social-media companies didn’t do more, sooner, to stop the threat.

It’s a question that Facebook, Twitter, YouTube, and others have faced before, going back to 2014 when the problem was content from extremists of a different sort: violent jihadist groups such as ISIS.

Since then, many social-media giants have developed technological and policy-based ways to help prevent extremist content from proliferating across their sites and even to help law enforcement better track potential violent actors. But those efforts were aimed at foreign Islamic extremists, not domestic threats.

Monika Bickert, Facebook’s head of product policy and counterterrorism, has helped her company move farther along in this regard than some others. 

What role did Facebook play in the events that unfolded last week? A limited one: Robert Bowers, the charged Pittsburgh shooter, kept his threats and violent posts to a relatively obscure right-wing platform called Gab. Soon after he joined Gab in January, he began to post and spread images and content threatening to Jews.

Cesar Sayoc had a Facebook profile that he used to advance conspiracy theories. He also threatened people on Twitter, such as political analyst Rochelle Ritchie. Ritchie reported the threats to the platform. Last weekend, the company apologized for failing to act sooner.

Although Sayoc had a small presence on Facebook, the company might still have had a lot of information about him. Facebook monitors extremist rhetoric and content, of the Islamist variety, on sites that aren’t Facebook. It employs contractors to watch extremist chat rooms and other places so they can be ready to identify and tag threatening language, images, and content on Facebook.

Facebook’s Erin Marie Saltman, a policy manager at Facebook who oversees counterterrorism efforts in Europe, Africa, and the Middle East, disclosed this at the GLOBSEC security summit in May.

“There are a lot of people in other parts of the world that are not Facebook and not government,” Saltman said. “They are intel providers that sit and squat on a lot of these other sites and they tell us, in as close to real time as possible, when bad content is being released and so we know about it as soon as possible. 

So when the Abu Bakr al-Baghdadi speech was released a little while ago, and it wasn’t in video form, just audio, we were able to hash[tag] it before it started hitting our site.”

Predicting potentially violent behavior requires as much digitally collected data as possible, precisely the sort of data that intel vendors watching sites like Gab might notice. But when Defense One asked Facebook representatives whether they monitor sites like Gab for such content, or potential indicators of violence, they declined to say.

“As Erin mentioned, we work with intel and research firms who monitor many platforms, but we prefer not to disclose further details as bad actors actively work to circumvent our detection techniques,” a Facebook spokesperson said. 

“Since the bombing attempts, and the shooting in Pittsburgh, teams across our company have been monitoring developments in real time to understand both situations and how they relate to content on our site,” they added.

In 2011, J. Reid Meloy, a forensic psychologist and consultant to FBI’s Behavioral Analysis Units at Quantico, identified eight behaviors that can predict lone-wolf attacks based on ideological extremism. Sayoc and Bowers exhibited several of them across multiple social media sites. 

If social-media companies could search for these subtle indicators of a potentially dangerous person, behaviors, such as fixation or obsession, in the context of overtly troubling posts and comments such as direct threats, patterns could emerge to predict an individual’s behavior.

Cross-platform analysis of individuals’ data residue is what contemporary micro-targeting for advertisements is based on. It works to predict whether a person might be open to a specific product pitch but it also works to predict potentially harmful behavior. 

Facebook is already using AI to spot suicidal tendencies signaled by text patterns. The same algorithms could be applied to spot violent extremism, as could network analysis and even semantic text analysis. 

That information, coupled with the identification of violent messages or threats spread on other sites, could go a long way toward predicting and preventing violent behavior and the posting of extremist content. And it is, but it’s mostly violent Islamic behavior and content.

Consider the case of Demetrius Nathaniel Pitts, a Cleveland man recently charged with plotting a jihadist-inspired terrorist attack. Authorities monitored Pitts’s Facebook posts carefully after he commented on a photo of an al-Qaida training camp. 

His posts extorted Muslims to learn how to operate firearms, posts that law enforcement officials described as “disturbing” to USA Today. But pages urging non-Muslims (or people who are not explicitly Muslim) to own and practice with firearms are common on Facebook.

“We continually enforce our Community Standards through a combination of technology, reports from our community, and human review. This includes our hate speech policy that prohibits content that attacks people based on their race, ethnicity, national origin, religious affiliations, sexual orientation, caste, sex, gender, gender identity, and serious disease or disability,” said the spokesperson.

Following the public outcry against the proliferation of jihadist extremist messaging, Facebook and other sites tried a technique called hashing: essentially, to mark Islamic extremist content as individuals tried to spread it from one site to another. In 2016, Facebook executives led an effort to share data on hashed images across platforms.

“It creates the equivalent of a digital fingerprint so you can know when these things are coming up. We encourage that type of sharing, the hash sharing. Anybody using types of video, photo matching, would be able to use the hashes we are trying to share,” said Saltman.

Could hashed images and data from accounts like Bowers’s and Sayoc’s be relevant to law enforcement? Potentially, but the practice of hash sharing doesn’t involve the government, said Saltman. Instead, she said, the goal was to make a “safe tech space” for technology platforms to use whatever tools they saw fit.

“This is a by-industry, for-industry effort; it doesn’t include government or NGOs. It’s really so we can create a safe space so that some of these smaller platforms that are really scared about talking outside of industry, and admitting you have a problem is step one, can come together in a safe tech space and start operationalising around some of this.”

In a conversation with New York Times reporters on Sunday, Gab founder Andrew Torba denied that he or any Gab employer should monitor content on the site. 

“Twitter and other platforms police ‘hate speech’ as long as it isn’t against President Trump, white people, Christians, or minorities who have walked away from the Democratic Party,” he wrote. “This double standard does not exist on Gab.”

Defense One:

You Might Also Read:

A Genocide Incited On Facebook:

The Weaponization Of Social Media

« British Policing Faces The Future
Blockchain Can Improve Manufacturing »

CyberSecurity Jobsite
Perimeter 81

Directory of Suppliers

CYRIN

CYRIN

CYRIN® Cyber Range. Real Tools, Real Attacks, Real Scenarios. See why leading educational institutions and companies in the U.S. have begun to adopt the CYRIN® system.

ON-DEMAND WEBINAR: What Is A Next-Generation Firewall And Why Does It Matter

ON-DEMAND WEBINAR: What Is A Next-Generation Firewall And Why Does It Matter

See how to use next-generation firewalls (NGFWs) and how they boost your security posture.

Cyber Security Supplier Directory

Cyber Security Supplier Directory

Our Supplier Directory lists 6,000+ specialist cyber security service providers in 128 countries worldwide. IS YOUR ORGANISATION LISTED?

Authentic8

Authentic8

Authentic8 transforms how organizations secure and control the use of the web with Silo, its patented cloud browser.

NordLayer

NordLayer

NordLayer is an adaptive network access security solution for modern businesses — from the world’s most trusted cybersecurity brand, Nord Security. 

vArmour

vArmour

vArmour is the industry’s first distributed security system that provides insight and control for multi-cloud environments.

Clusit

Clusit

Clusit is the Italian Association for Information Security, a nonprofit organization devoted to promoting every aspect of information security.

Secardeo

Secardeo

Secardeo is a provider of corporate solutions using digital signatures and certificates. Our solutions enable the user transparent end-to-end encryption of e-mails between organizations.

ClearDATA

ClearDATA

The ClearDATA Managed Cloud protects sensitive healthcare data using purpose-built DevOps automation, compliance and security safeguards, and healthcare expertise.

Data Resolve Technologies

Data Resolve Technologies

Data Resolve offer a mechanism through which customers can detect and tackle various kinds of sensitive activities pertaining to data loss and data theft.

Trapezoid

Trapezoid

Trapezoid is a cybersecurity company developing Firmware Integrity Management solutions designed to detect unauthorized changes to firmware & BIOS across the entire data center infrastructure.

Real Random

Real Random

Real Random is on a mission to enhance existing and new crypto-systems with its revolutionary solution to generating numbers that are Truly Random.

GlobalPass

GlobalPass

Covering 200+ countries with 78 000 databases, GlobalPass provides sophisticated facial biometrics verification and deep screening, delivering peace of mind to every client.

Rede Nacional CSIRT

Rede Nacional CSIRT

Rede Nacional CSIRT is a national network of CSIRTs in Portugal aimed at cooperation and mutual assistance in the handling of incidents and in the sharing of good security practices.

Exabeam Cyberversity

Exabeam Cyberversity

Exabeam Cyberversity is a philanthropic program to help aspiring cybersecurity professionals navigate career options and increase industry-wide diversity through knowledge sharing and networking.

Ermetic

Ermetic

Ermetic’s identity-first cloud infrastructure security platform provides holistic, multi-cloud protection in an easy-to-deploy SaaS solution.

Green Radar

Green Radar

Green Radar is a next generation cybersecurity company which combines technologies and services together to deliver Threat Detection for Emails and Deep Threat Analytics and Response.

Surfshark

Surfshark

Surfshark is a cybersecurity company focused on developing humanized privacy & security protection solutions to secure people's digital lives.

Aravo Solutions

Aravo Solutions

Your Extended Enterprise is full of hidden risks – Aravo makes them visible, measurable, and manageable.

Xcelerate Solutions

Xcelerate Solutions

Xcelerate Solutions is a leading defense and national security company, providing integrated solutions in three service areas – Enterprise Security, Digital Transformation, and Strategic Consulting.

Theori

Theori

Theori tackles the most difficult cybersecurity challenges from an attacker’s perspective and conquers them as the best strategic security experts.