The Dark Side Of The New Dawn In AI

As ChatGPT’s active users grew to 100 million in just two months fascinated by what it can do, it has already started to reveal its dark side. It has become the cause of security breaches and risked violation of privacy, compliance and governance regulations; not through attacks, but users are voluntarily (and unlawfully) uploading sensitive information to the system in order to generate insights.

One report found that over 4% of employees have already tried to put sensitive company data into the model. The recent release of GPT4, which uses Large Language Models (LLM), can accept much larger chunks of text and is likely to make this problem much worse (and quickly). 

The report came from a company that detected and blocked the 67,000 attempts of misuse across their client base. Most organizations don’t have this capability. One executive tried to paste the corporate strategy into the system to make a PowerPoint, and a doctor put in a patient’s name and medical condition to draft a report.
Cyber experts have proven that training data extraction attacks are possible in GPT, where an attacker can get the system to recall, verbatim, sensitive information it has been given. 

What Are The Shortfalls Of ChatGPT In Cyber Security?

As well as potentially having your spilled data ‘hacked’ out of GPT’s databases, just spilling it in the first place could be a breach of many different security policies, secrecy laws, and privacy regulations. And on the flip side, retrieving and using someone else’s information from GPT, that turns out to be proprietary, confidential, or copyright, could also get your company in trouble. 

The only way to stop this, apart from blocking access to GPT or other LLM tools, is training and education of humans using the technology. But it’s difficult to train every staff member, and even more challenging to make sure they understand and retain that training. It’s even harder to make sure they apply the training daily and consistently to ensure they are not exposing their employer to significant risks.  

The rise of chatbots and reliance on a machine to provide answers means we never know when we are being given the correct information.  

Chatbots have the challenge of being opaque. When they give an answer, it’s hard to fact check that answer. We run the risk of solely relying on a machine’s recommendation, even when that recommendation may be wrong.

Now let’s look at this issue in light of cybersecurity. If we ask a machine security-related questions and the answer is potentially incorrect, the consequences of being wrong can be catastrophic. 

This is why it’s critical not to rely on a black-box, algorithmic AI for regulatory or security compliance. When we are deciding what law or policy to apply, we need to be able to understand and challenge the evidence behind that decision.

Safeguarding From Emerging ChatGPT Threats

Phishing is already one of the most common and successful attack methods for bad actors. ChatGPT can put the ability to craft more believable phishing messages with ease in the wrong hands. Deepfakes are the next level. A believable email from your boss asking you to email a sensitive document, followed up by a video call that looks and sounds exactly like her? These are some of the enormous challenges we face that training alone can’t control.  

Having a data spill is essentially inevitable. We can never reduce the likelihood of a breach to zero, because we will always have trusted insiders. The approach to take now is to reduce the potential impact of a future breach.

Know what data you have, what risk it has, and what value. What rules apply to it, where it is, who is doing what to it. And know what needs to be locked down, and what can be disposed of (across the whole enterprise). This is something we can use AI for right now, and it’s really moving the needle back to the side of good governance. 

Rachael Greaves is CEO/CSO and Founder of Castlepoint Systems

You Might Also Read:

ChatGPT Language Model Risks:

___________________________________________________________________________________________

If you like this website and use the comprehensive 6,500-plus service supplier Directory, you can get unrestricted access, including the exclusive in-depth Directors Report series, by signing up for a Premium Subscription.

  • Individual £5 per month or £50 per year. Sign Up
  • Multi-User, Corporate & Library Accounts Available on Request

 


Cyber Security Intelligence: Captured Organised & Accessible


 

« If It’s Convenient Be Suspicious – The Human Aspect
Britain's National Cyber Force Reveals Its Operating Doctrine »

ManageEngine
CyberSecurity Jobsite
Check Point

Directory of Suppliers

ZenGRC

ZenGRC

ZenGRC (formerly Reciprocity) is a leader in the GRC SaaS landscape, offering robust and intuitive products designed to make compliance straightforward and efficient.

ManageEngine

ManageEngine

As the IT management division of Zoho Corporation, ManageEngine prioritizes flexible solutions that work for all businesses, regardless of size or budget.

Lunar

Lunar

Lunar is a free, enterprise-grade, compromised-credentials monitoring platform, available to every company, everywhere.

Oppida

Oppida

Oppida provides tailored IT security services to help you identify security gaps and assist in finding the most effective remediation.

AM-CERT

AM-CERT

AM-CERT is the national Computer Emergency Response Team for Armenia.

S2 Grupo

S2 Grupo

S2 Grupo is the benchmark company in Europe and Latin America, for Cyber Intelligence and mission critical systems operations.

Equilibrium Security Services

Equilibrium Security Services

Equilibrium Security Services is a specialist cyber security company providing a full spectrum of IT security solutions from consultancy to design & implementation and managed security services.

Raytheon Technologies

Raytheon Technologies

Raytheon Intelligence & Space delivers solutions that protect every side of cyber for government agencies, businesses and nations.

Avansic

Avansic

Avansic is a leading provider of e-discovery and digital forensics services to attorneys, litigation support teams, and business communities.

Myra Security

Myra Security

Myra's platform protects websites, web applications and APIs against DDoS attacks and malicious traffic, with WAF and CDN for fast delivery.

Cyber Defense Agency (CDA)

Cyber Defense Agency (CDA)

Cyber Defense Agency is a premier professional services firm specializing in cyber security, computer network defense, and information security.

Tempest

Tempest

TEMPEST is a leading provider of IT products and services including solutions for network and application security.

Randori

Randori

Randori is an attack platform that provides "red-teaming" as a service - basically, staging simulated hack attacks to test for vulnerabilities and gaps in the security response.

AXA XL

AXA XL

AXA XL is the P&C and Specialty Risk Division of AXA. Professional insurance products include Cyber Insurance.

Splone

Splone

Splone is a Berlin-based IT security research team and consultancy. We help improve IT-security by offering red team assements, penetration tests, audits and customized consulting.

ECHO Project

ECHO Project

The main objective of ECHO is to strengthen the cyber defence of the European Union, enhancing Europe’s technological sovereignty through effective and efficient multi-sector collaboration.

MorganFranklin Consulting

MorganFranklin Consulting

MorganFranklin Consulting is a management advisory firm that works with businesses and government to address complex and transformational technology and business objectives including cybersecurity.

Viettel Cyber Security

Viettel Cyber Security

Viettel Cyber Security is an organization under the Military Telecommunication Industry Group, conducting research and developing information security solutions for domestic and foreign customers.

Sentar

Sentar

Sentar is a cyber intelligence company, applying advanced analytics and systems engineering expertise to protect our national security by securing mission-critical assets.

Archer & Round

Archer & Round

Archer & Round delivers industry-leading cyber security solutions designed to fortify businesses against cyber risks.

Security Architect

Security Architect

Security Architect is a leading provider of IT and cybersecurity solutions, at the forefront of technological protection and innovation for enterprise companies.