Future Phishing Attacks Will Use Generative Machine Learning

Phishing attacks aim to steal confidential information using sophisticated methods through a series of techniques via content injection, social engineering, online social networks, and mobile applications. More than 85% of credential phishing attacks looked like legitimate common business workflows to trick end-users into engaging with the email.

Recent advances in Deep Learning have enabled improvements in generative model architectures, and some state-of-the-art models can now produce outputs realistic enough to fool humans.

Now, criminal data analysts and technicians have decided to use Generative Machine Learning (GML) models that learn the distribution of data from a sample dataset and can then generate new attack vectors. 

Until recently with a phishing attack, the victims had an advantage as criminals have had to do everything by hand. All of the text detail, from the email attempt to lure the victim takes time to create and if you watched out for them the less sophisticated attacks are easy to see.

To avoid and mitigate the risks of these attacks, several phishing detection approaches were developed, among which deep learning algorithms provided promising results. However, the results and the corresponding lessons learned are fragmented over many different studies and there is a lack of a systematic overview of the use of deep learning algorithms in phishing detection and now some criminals are using the Deep Learning language model GPT.

OpenAI's version of GPT has demonstrated that incredibly powerful Machine Learning (ML) text generation can also be designed to be quite simple for lay programmers to implement. More recently, OpenAI's Dall-E was used to demonstrate that creating a realistic fake image can be as simple as calling a function with a brief language description.

"Think about something as simple as an image of a grocery bag with a fake logo. If you wanted to get this kind of thing a few years ago, you would have needed to pay someone who knows how to do Photoshop to make the logo and create some fake image. Now this whole process has been boiled down to just one single line of English text," said Prashanth Arun, head of data science for Armorblox.

"Imagine you make a fake Candle Company, with an entire range of candles with your little logo and product descriptions that say different things, it gives you a sense that, you know, these guys have been around for a long time," said Arun.

Future phishing attacks will come with detailed web presences and will be generated with the click of a button.

The idea that GML creates security problems is not new, although one of the problems with it is that it was good at short pieces of text, but the text tended to become unstable once messages become too long.

That type of systemic problem could even be gamed by someone looking to poison data collection. If Planters gave Mr. Peanut an employee bio as chief snack officer, that could translate to a business email compromise campaign where Mr. Peanut requests invoices be paid. Nevertheless, attacks like this can be difficult to defend against and the same problems discerning facts that troubled the phishing ML to would also plague the defensive ML. 

The combination of ease of use and difficulty of defence could mean generative attacks make a substantial change in the threat landscape sooner than most defenders would be prepared for.

"For high-value targets, I think it's still going to be humans running the attacks, simply because the ROI on such scams are much higher," said Arun. "But for a lot of these spray and pray kinds of spammy stuff, I think the quality of that is going to be improved significantly."

Phish Protection:    Mahmood & Dabassi:     SC Magazine:   SC Magazine:   Research Gate

Research Gate:     CPS-VO:      Springer:    

You Might Also Read: 

A Phishing Attack That Delivers Three Forms Of Malware:

 

« Edge AI: The Future of Artificial Intelligence And Edge Computing
CISA Detects Many New Cyber Security Vulnerabilities »

Infosecurity Europe
CyberSecurity Jobsite
Perimeter 81

Directory of Suppliers

ManageEngine

ManageEngine

As the IT management division of Zoho Corporation, ManageEngine prioritizes flexible solutions that work for all businesses, regardless of size or budget.

Infosecurity Europe, 3-5 June 2025, ExCel London

Infosecurity Europe, 3-5 June 2025, ExCel London

This year, Infosecurity Europe marks 30 years of bringing the global cybersecurity community together to further our joint mission of Building a Safer Cyber World.

IT Governance

IT Governance

IT Governance is a leading global provider of information security solutions. Download our free guide and find out how ISO 27001 can help protect your organisation's information.

North Infosec Testing (North IT)

North Infosec Testing (North IT)

North IT (North Infosec Testing) are an award-winning provider of web, software, and application penetration testing.

Jooble

Jooble

Jooble is a job search aggregator operating in 71 countries worldwide. We simplify the job search process by displaying active job ads from major job boards and career sites across the internet.

Intrinsic-ID

Intrinsic-ID

Intrinsic-ID's authentication technology creates unique IDs and keys to authenticate chips, data, devices and systems.

CipherPoint Software

CipherPoint Software

CipherPoint Software provides data-centric auditing and protection solutions for securing unstructured information

MAY Cyber Technology

MAY Cyber Technology

MAY Cyber Technology is a Security Management solutions provider located in Turkey & Germany.

Seqrite

Seqrite

Seqrite offers a highly advanced range of enterprise and IT security solutions to protect your organization's most critical data.

Sistem Integra (SISB)

Sistem Integra (SISB)

SISB provide IT Security Infrastructure & Development, Mechanical & Electrical Services, Fire Safety & Detection Services, Facilities Management & Application Development.

Epati Information Technologies

Epati Information Technologies

ePati Information Technologies is a specialist in information technology and cyber security.

ProWriters

ProWriters

As a leading cyber insurance company, ProWriters offers flexible Cyber Liability Insurance coverage designed to cover privacy, data, and network exposures.

CACI International

CACI International

CACI is at the forefront of developing and delivering technological breakthroughs that transform and optimize government operations.

NetWitness

NetWitness

NetWitness empowers security teams to rapidly detect today’s targeted and sophisticated attacks with unparalleled visibility.

Finesse Global

Finesse Global

Finesse is a global system integration and digital business transformation company.

Vali Cyber

Vali Cyber

Vali Cyber was founded in 2020 with the mission of addressing the specific cybersecurity needs of Linux.

Allot

Allot

Allot are a global provider of leading innovative network intelligence and security solutions for Service Providers and Enterprises worldwide.

Sequentur

Sequentur

Sequentur is an award-winning Managed IT Services company. We are SOC 2 certified and provide Managed IT Services and Cybersecurity services to businesses nationwide.

CyberSecureRIA

CyberSecureRIA

We founded CyberSecureRIA specifically to secure and support RIAs. We exist to secure SEC-registered RIAs, and keep them compliant with cybersecurity regulations.

Avatar Managed Services

Avatar Managed Services

Avatar offers proven, process driven IT support to companies who want to utilize their technology to their best advantage.

Blackwired

Blackwired

Blackwired has established a new category in cyber security with an intelligence-led model based on the USMC’s Combat Hunter programme ‘Left of Bang’.