The Dark Side Of The New Dawn In AI

As ChatGPT’s active users grew to 100 million in just two months fascinated by what it can do, it has already started to reveal its dark side. It has become the cause of security breaches and risked violation of privacy, compliance and governance regulations; not through attacks, but users are voluntarily (and unlawfully) uploading sensitive information to the system in order to generate insights.

One report found that over 4% of employees have already tried to put sensitive company data into the model. The recent release of GPT4, which uses Large Language Models (LLM), can accept much larger chunks of text and is likely to make this problem much worse (and quickly). 

The report came from a company that detected and blocked the 67,000 attempts of misuse across their client base. Most organizations don’t have this capability. One executive tried to paste the corporate strategy into the system to make a PowerPoint, and a doctor put in a patient’s name and medical condition to draft a report.
Cyber experts have proven that training data extraction attacks are possible in GPT, where an attacker can get the system to recall, verbatim, sensitive information it has been given. 

What Are The Shortfalls Of ChatGPT In Cyber Security?

As well as potentially having your spilled data ‘hacked’ out of GPT’s databases, just spilling it in the first place could be a breach of many different security policies, secrecy laws, and privacy regulations. And on the flip side, retrieving and using someone else’s information from GPT, that turns out to be proprietary, confidential, or copyright, could also get your company in trouble. 

The only way to stop this, apart from blocking access to GPT or other LLM tools, is training and education of humans using the technology. But it’s difficult to train every staff member, and even more challenging to make sure they understand and retain that training. It’s even harder to make sure they apply the training daily and consistently to ensure they are not exposing their employer to significant risks.  

The rise of chatbots and reliance on a machine to provide answers means we never know when we are being given the correct information.  

Chatbots have the challenge of being opaque. When they give an answer, it’s hard to fact check that answer. We run the risk of solely relying on a machine’s recommendation, even when that recommendation may be wrong.

Now let’s look at this issue in light of cybersecurity. If we ask a machine security-related questions and the answer is potentially incorrect, the consequences of being wrong can be catastrophic. 

This is why it’s critical not to rely on a black-box, algorithmic AI for regulatory or security compliance. When we are deciding what law or policy to apply, we need to be able to understand and challenge the evidence behind that decision.

Safeguarding From Emerging ChatGPT Threats

Phishing is already one of the most common and successful attack methods for bad actors. ChatGPT can put the ability to craft more believable phishing messages with ease in the wrong hands. Deepfakes are the next level. A believable email from your boss asking you to email a sensitive document, followed up by a video call that looks and sounds exactly like her? These are some of the enormous challenges we face that training alone can’t control.  

Having a data spill is essentially inevitable. We can never reduce the likelihood of a breach to zero, because we will always have trusted insiders. The approach to take now is to reduce the potential impact of a future breach.

Know what data you have, what risk it has, and what value. What rules apply to it, where it is, who is doing what to it. And know what needs to be locked down, and what can be disposed of (across the whole enterprise). This is something we can use AI for right now, and it’s really moving the needle back to the side of good governance. 

Rachael Greaves is CEO/CSO and Founder of Castlepoint Systems

You Might Also Read:

ChatGPT Language Model Risks:

___________________________________________________________________________________________

If you like this website and use the comprehensive 6,500-plus service supplier Directory, you can get unrestricted access, including the exclusive in-depth Directors Report series, by signing up for a Premium Subscription.

  • Individual £5 per month or £50 per year. Sign Up
  • Multi-User, Corporate & Library Accounts Available on Request

 


Cyber Security Intelligence: Captured Organised & Accessible


 

« If It’s Convenient Be Suspicious – The Human Aspect
Britain's National Cyber Force Reveals Its Operating Doctrine »

CyberSecurity Jobsite
Perimeter 81

Directory of Suppliers

Cyber Security Supplier Directory

Cyber Security Supplier Directory

Our Supplier Directory lists 6,000+ specialist cyber security service providers in 128 countries worldwide. IS YOUR ORGANISATION LISTED?

CSI Consulting Services

CSI Consulting Services

Get Advice From The Experts: * Training * Penetration Testing * Data Governance * GDPR Compliance. Connecting you to the best in the business.

Clayden Law

Clayden Law

Clayden Law advise global businesses that buy and sell technology products and services. We are experts in information technology, data privacy and cybersecurity law.

Syxsense

Syxsense

Syxsense brings together endpoint management and security for greater efficiency and collaboration between IT management and security teams.

Jooble

Jooble

Jooble is a job search aggregator operating in 71 countries worldwide. We simplify the job search process by displaying active job ads from major job boards and career sites across the internet.

Cloud Credential Council (CCC)

Cloud Credential Council (CCC)

The CCC is a leading provider of vendor-neutral certification programs that empower IT and business professionals in their digital transformation journey.

Allegro Software

Allegro Software

Allegro provide secure software for the Internet of Things.

mmCERT

mmCERT

mmCERT is the national Computer Emergency Response Team for Myanmar.

Nixu

Nixu

Nixu is the largest Nordic specialist company in information security consulting.

North American Electric Reliability Corporation (NERC)

North American Electric Reliability Corporation (NERC)

NERC is a not-for-profit international regulatory authority whose mission is to assure the reliability and security of the bulk power system in North America.

ERNW

ERNW

ERNW is an independent IT Security service provider with a focus on consulting and testing in all areas of IT security.

ENLIGHTENi

ENLIGHTENi

ENLIGHTENi are the platform to develop next-gen talent in Technology, Risk, and Cybersecurity. Our mission is to develop next-gen talent through challenge-based learning and team collaboration.

CyberSwarm

CyberSwarm

CyberSwarm is developing a neuromorphic System-on-a-Chip dedicated to cybersecurity which helps organizations secure communication between connected devices and protect critical business assets.

Heidrick & Struggles International

Heidrick & Struggles International

Heidrick & Struggles is a premier provider of leadership consulting and senior-level executive search services for roles including Information & Technology Officers and Cybersecurity.

Stamus Networks

Stamus Networks

Stamus Networks offers Scirius Security Platform solutions that marry real-time network traffic data with enhanced Suricata intrusion detection (IDS) and an advanced analytics engine.

Raonsecure

Raonsecure

Raonsecure is one of Korea’s leading ICT security software companies – providing a variety of PC and mobile security solutions to financial institutions, government, and enterprise.

VirtualArmour

VirtualArmour

VirtualArmour is a managed security services provider with global reach and local attitude.

Aigner Business Solutions

Aigner Business Solutions

Aigner Business Solutions GmbH is a specialist in IT-Security and Data Protection. Concise and focussed.

RegScale

RegScale

RegScale helps organizations comply in real-time with multiple compliance requirements (NIST, CMMC, ISO, SOX, etc), scalable to meet the needs of the entire enterprise.

Island

Island

Island puts the enterprise in complete control of the browser, delivering a level of governance, visibility, and productivity that simply weren’t possible before.

SecureDNE

SecureDNE

SecureDNE are a leading provider of cutting-edge Fractional CISO, Managed Cybersecurity Services, and Cybersecurity Engineering Solutions.