ChatGPT Language Model Risks

ChatGPT has exploded across the Internet and has created a new era of Artificial Intelligence (AI).  With AI tools becoming increasingly powerful, the question many leaders are exploring is how to use these tools in our businesses.

AI chatbots and Large Language Models (LLMs) present a rising security threat, the British National Cyber Security Agency (NCSC) has warned. The NCSC has issued a detailed  warning advising people to “take great care” with data they choose to submit to chatbots, given companies will “almost certainly” access it.

ChatGPT, the popular chatbot from OpenAI, is estimated to have reached 100 million monthly active users just two months after launch. 

It is a fast-growing application and its popularity is leading many competitors to develop their own services and models, or to rapidly deploy those that they’ve been developing internally. However, as the use of AI-powered language models such as ChatGPT becomes more prevalent in both business and personal settings, it's critical to understand the serious cyber security risks they present. 

They are powerful tools, but there are very real dangers to consider, as well as ethical implications, especially if you plan to use them in your business. 

What Are ChatGPT & LLMs?

ChatGPT is an Artificial Intelligence Chatbot developed by OpenAI, a US tech startup. It's based on GPT-3, a language model released in 2020 that uses deep-learning to produce human-like text, but the underlying LLM technology has been around much longer.

An LLM is where an algorithm has been trained on a large amount of text-based data, typically scraped from the open Internet, and so covers web pages and, depending on the LLM, other sources such as scientific research, books or social media posts. This covers such a large volume of data that it’s not possible to filter all offensive or inaccurate content at ingest, and so 'controversial' content is likely to be included in its model.

They use algorithms to analyse the relationships between different words and turn that into a probability model. It is then possible to give the algorithm a 'prompt' - by asking it a question, for example - and it will provide an answer based on the relationships of the words in its model.

Typically, the data in its model is static after it has been trained, although it can be refined by 'fine-tuning'which is training on additional data and 'prompt augmentation' which is providing context information about the question. 

ChatGPT allows users to ask an LLM questions, as you would when holding a conversation with a chatbot. Other current examples of LLMs include Google’s Bard and Meta’s LLaMa.

 LLMs are impressive for their ability to generate a huge range of convincing content in multiple human and computer languages, however, they contain some serious flaws. According to the NCSC:

  • They can get things wrong and ‘hallucinate’ incorrect facts.
  • They can be biased, are often gullible (in responding to leading questions, for example).
  • They require very large and expensive computer resources and access to vast data to train from scratch.
  • They can be coaxed into creating toxic content and are prone to ‘injection attacks’.

LLMs Could Reveal Your Information

A common concern is that an LLM might 'learn' from your prompts and offer that information to others who query for related things. Currently, LLMs are trained, and then the resulting model is queried. An LLM does not (as of writing) automatically add information from queries to its model for others to query. That is, including information in a query will not result in that data being incorporated into the LLM. However, the query will be visible to the organisation providing the LLM - as in the case of ChatGPT, to OpenAI. Those queries are stored and will almost certainly be used for developing the LLM service or model at some point. 

This could mean that the LLM provider and its business partners are able to read queries and may incorporate them into future versions. Consequently, the terms of use and privacy policy need to be thoroughly understood before asking sensitive questions.

A question might be sensitive because of data included in the query, or because who is asking the question. An example might be if a CEO is discovered to have asked 'how best to lay off an employee?', or somebody asking revealing health or relationship questions. There is also the possibility of the aggregation of information across multiple queries using the same login.

Another risk, which increases as more organisations produce LLMs, is that queries stored online may be hacked, leaked, or more likely accidentally made publicly accessible. This could include potentially user-identifiable information. A further risk is that the operator of the LLM is later acquired by an organisation with a different approach to privacy than was the case when users first entered the data.

The NCSC Recommends

  • Do not to include sensitive information in queries to public LLMs
  • Do not to submit queries to public LLMs that would lead to issues were they made public

How can you safely provide LLMs with sensitive information?

In the wake of the excitement around LLMs, many organisations may be wondering if they can use LLMs to automate certain business tasks, which may involve providing sensitive information either through fine-tuning or prompt augmentation. Whilst this approach is not recommended for public LLMs, ‘private LLMs’ might be offered by a cloud provider (for example), or can be entirely self hosted:

  • For cloud-provided LLMs, the terms of use and privacy policy again become key (as they are for public LLMs), but are more likely to fit within the existing terms for the cloud service. Organisations need to understand how the data they use for fine-tuning or prompt augmentation is managed.
    • Is it available to the vendor’s researchers or partners?
    • If so, in what form? Is data shared in isolation or in aggregation with other organisations?
    • Under what conditions can an employee at the provider view queries?
  • Self-hosted LLMs are likely to be highly expensive, however, following a security assessment they may be appropriate for handling organisational data. 

LLMs make life easier for Cyber Criminals

There have been some examples of how LLMs can help write malware. The concern is that an LLM might help someone with malicious intent (but insufficient skills) to create tools they would not otherwise be able to deploy. 
In their current state, LLMs suffer from appearing convincing and are suited to simple tasks rather than complex ones. This means LLMs are useful for 'helping experts save time', as the expert can validate the LLM's output.

For more complex tasks, it's currently easier for an expert to create the malware from scratch, rather than having to spend time correcting what the LLM has produced. However, an expert capable of creating highly capable malware is likely to be able to coax an LLM into writing capable malware.

This trade-off between 'using LLMs to create malware from scratch' and 'validating malware created by LLMs' will change as LLMs improve.

LLMs can also be queried to advise on technical problems. There is a risk that criminals might use LLMs to help with cyber attacks beyond their current capabilities, especially once an attacker has accessed a network. For example, if an attacker is struggling to escalate privileges or find data, they might ask an LLM, and receive an answer that's not unlike a search engine result, but with more context. 

Current LLMs provide convincing-sounding answers that may only be partially correct, particularly as the topic gets more niche. These answers might help criminals with attacks they couldn't otherwise execute, or they might suggest actions that hasten the detection of the criminal. In any case, the attacker’s queries will likely be stored and retained by LLM operators.

As LLMs improve there is a risk of criminals using LLMs to write convincing phishing emails, including emails in multiple languages. This may aid attackers with high technical capabilities but who lack linguistic skills, by helping them to create convincing phishing emails (or conduct social engineering) in the native language of their targets. Consequenty, the NCSC suggest that we might soon see:  

  • More convincing phishing emails as a result of LLMs.
  • Attackers trying techniques they didn't have familiarity with previously.
  • A risk of a lesser-skilled attacker writing highly capable malware.

Conclusion

LLMs, and ChatGPT are exciting developments with dynamic potential to engage users and gain despread acceptance. But, there are risks involved in the unrestricted use of public LLMs. Individuals and organisations should take great care with the data they choose to submit in prompts. The NCSC advise that users should ensure that those who want to experiment with LLMs are able to, but in a way that doesn't place an organisation's data at risk.

AI language models like ChatGPT offer incredible potential for businesses and individuals, but they also present serious security and ethical risks that must be addressed. By following best practices and taking proactive steps to mitigate the risks, the safe and responsible use of these tools can be ensured.

NCSC:     Reuters:     TechRadar:     Proactive Investors:     Maddyness

You Might Also Read: 

The Dark Side Of AI:

___________________________________________________________________________________________

If you like this website and use the comprehensive 6,500-plus service supplier Directory, you can get unrestricted access, including the exclusive in-depth Directors Report series, by signing up for a Premium Subscription.

  • Individual £5 per month or £50 per year. Sign Up
  • Multi-User, Corporate & Library Accounts Available on Request

Cyber Security Intelligence: Captured Organised & Accessible


 

« Russia's Disinformation Campaign Targets Ukraine's Supporters
World Backup Day  »

CyberSecurity Jobsite
Perimeter 81

Directory of Suppliers

NordLayer

NordLayer

NordLayer is an adaptive network access security solution for modern businesses — from the world’s most trusted cybersecurity brand, Nord Security. 

Syxsense

Syxsense

Syxsense brings together endpoint management and security for greater efficiency and collaboration between IT management and security teams.

Jooble

Jooble

Jooble is a job search aggregator operating in 71 countries worldwide. We simplify the job search process by displaying active job ads from major job boards and career sites across the internet.

CYRIN

CYRIN

CYRIN® Cyber Range. Real Tools, Real Attacks, Real Scenarios. See why leading educational institutions and companies in the U.S. have begun to adopt the CYRIN® system.

North Infosec Testing (North IT)

North Infosec Testing (North IT)

North IT (North Infosec Testing) are an award-winning provider of web, software, and application penetration testing.

IONU Security

IONU Security

IONU offer a security platform focused specifically on providing Data-centric Security.

Seclore

Seclore

Seclore is the most advanced, secure, and automated Enterprise Digital Rights Management (EDRM) solution available.

Marsh

Marsh

Marsh is a global leader in insurance broking and risk management and has been a leader in combatting cyber threats since their emergence.

7Safe

7Safe

7Safe has been delivering hands-on digital security training courses since 2001 and offer e a portfolio of university and industry-accredited courses.

Bufferzone Security

Bufferzone Security

Bufferzone is a patented containment solution that defends endpoints against advanced malware and zero-day attacks while maximizing user and IT productivity.

TorGuard

TorGuard

TorGuard is a Virtual Private Network services provider offering secure encrypted access to the internet.

Phew

Phew

Phew are New Zealand cyber security specialists with expertise and experience forged in global financial markets, IT&T, management consulting and SME business management.

MER Group

MER Group

MER Group is a world-leading integrator in the areas of communications and security. MER cyber solutions cover the entire range of cyber and intelligence related products and services.

Spanish Network of Excellence on Cybersecurity Research (RENIC)

Spanish Network of Excellence on Cybersecurity Research (RENIC)

RENIC is a membership based sectoral association that includes research centers and other agents of the research cybersecurity ecosystem in Spain.

PhishX

PhishX

PhishX is a SaaS platform for security awareness that simulates Cyberthreats, train people, while measure and analysis results, reducing Cybersecurity risks for People and Companies.

Luxembourg Office of Accreditation & Surveillance (OLAS)

Luxembourg Office of Accreditation & Surveillance (OLAS)

OLAS is the national accreditation body for Luxembourg. The directory of members provides details of organisations offering certification services for ISO 27001.

Trusted Objects

Trusted Objects

Trusted Object's mission is to provide state of the art security solutions and services enabling a strong root of trust for the IoT ecosystem.

Gorodissky IP Security

Gorodissky IP Security

Gorodissky IP Security is a comprehensive approach to protecting your intellectual property on the Internet and beyond.

Unit21

Unit21

Unit21 helps protect businesses against adversaries through a simple API and dashboard for detecting and managing money laundering, fraud, and other sophisticated risks across multiple industries.

ADGS

ADGS

ADGS is a deeptech company focused in the fields of Agent-Based simulations (Emergent Behavior), Cybersecurity and Biometrics, Social Dynamics, Natural Language Processing and Artificial Intelligence.

GreenPages Technology Solutions

GreenPages Technology Solutions

GreenPages provide expert strategic guidance and proven cloud-era solutions for our clients. Every day we help organizations leverage the cloud securely with less risk and cost.