Preventing The Hacked AI Apocalypse

Attacks are an increasingly worrisome threat to the performance of artificial intelligence applications.

If an attacker can introduce nearly invisible alterations to image, video, speech, and other data for the purpose of fooling AI-powered classification tools, it will be difficult to trust this otherwise sophisticated technology to do its job effectively.

Imagine how such attacks could undermine AI-powered autonomous vehicles ability to recognise obstacles, content filters’ effectiveness in blocking disturbing images, or in access systems’ ability to deter unauthorized entry.

Some people argue that adversarial threats stem from deep flaws in the neural net technology that powers today’s AI. After all, it’s well-understood that many machine learning algorithms are vulnerable to adversarial attacks.

However, you could just as easily argue that this problem calls attention to weaknesses in enterprise processes for building, training, deploying, and evaluating AI models.

None of these issues are news to AI experts. There is even a Kaggle competition focused right now on fending off adversarial AI.

It’s true that the AI community lacks any clear consensus on best practices for building anti-adversarial defenses into deep neural networks. But from what I see in the research literature and industry discussions, the core approaches from which such a framework will emerge are already crystallising.

Going forward, AI developers will need to follow these guidelines to build anti-adversarial protections into their applications:

Assume the possibility of adversarial attacks on all in-production AI assets

As AI is deployed everywhere, developers need to assume that their applications will be high-profile sitting ducks for adversarial manipulation.

AI exists to automate cognition, perception, and other behaviors that, if they produce desirable results, might merit the praise one normally associates with “intelligence.”

However, AI’s adversarial vulnerabilities might result in cognition, perception, and other behaviors, perhaps far worse than any normal human being would have exhibited under the circumstances.

Perform an adversarial risk assessment prior to initiating AI development

Upfront and throughout the life cycle of their AI apps, developers should frankly assess their projects’ vulnerability to adversarial attacks.

As noted in a 2015 research paper published by the IEEE, developers should weigh the possibility of unauthorised parties gaining direct access to key elements of the AI project, including the neural net architecture, training data, hyper-parameters, learning methodology, and loss function being used.

Alternatively, the paper shows, an attacker might be able to collect a surrogate dataset from the same source or distribution as the training data used to optimize an AI neural net model. This could provide the adversary with insights into what type of ersatz input data might fool a classifier model that was built with the targeted deep neural net.

In another attack approach described by the paper, even when the adversary lacks direct visibility into the targeted neural net and associated training data, attackers could exploit tactics that let them observe “the relationship between changes in inputs and outputs … to adaptively craft adversarial samples.”

Generate adversarial examples as a standard activity in the AI training pipeline

AI developers should immerse themselves in the growing body of research on the many ways in which subtle adversarial alterations may be introduced.

Data scientists should avail themselves of the growing range of open source tools, for generating adversarial examples to test the vulnerability of CNNs and other AI models. More broadly, developers should consider the growing body of basic research including those that aren’t directly focused on fending off cybersecurity attacks.

Recognise the need to rely on both human curators and algorithmic discriminators of adversarial examples

The effectiveness of an adversarial attack depends on its ability to fool your AI apps’ last line of defense.

Adversarial manipulation of an image might be obvious to the naked eye but still somehow fool a CNN into misclassifying it. Conversely, a different manipulation might be too subtle for a human curator to detect, but a well-trained discriminator algorithm in GAN may be able to pick it out without difficulty.

Build ensemble models that use a range of AI algorithms for detecting adversarial examples

Some algorithms may be more sensitive than others to the presence of adversary-tampered images and other data objects. For example, a scenario in which a shallow classifier algorithm might detect adversarial images better than a deeper-layered CNN. They also found that some algorithms are best suited for detecting manipulations across an entire image, while others may be better at finding subtle fabrications in one small section of an image.

One approach for immunizing CNNs from these attacks might be to add what Cornell University researcher Arild Nøkland calls an “adversarial gradient” to the back-propagation of weights during an AI model’s training process. It would be prudent for data science teams to test the relative adversary-detection advantages of different algorithms using ongoing A/B testing both in development and production environments.

Reuse adversarial-defense knowledge to improve AI resilience against bogus input examples

As noted in a 2016 research paper published by the IEEE, data scientists can use transfer-learning techniques to reduce the sensitivity of a CNN or other model to adversarial alterations in input images.

Whereas traditional transfer learning involves applying statistical knowledge from an existing model to a different one, the paper discusses how a model’s existing knowledge, gained through training on a valid data set, might be “distilled” to spot adversarial alterations.

According to the authors, “we use defensive distillation to smooth the model learned by a, distributed neural net, architecture during training by helping the model generalise better to samples outside of its training dataset.”

The result is that a model should be better able to recognise the difference between adversarial examples (those that resemble examples in its training set) and non-adversarial examples (those that may deviate significantly from those in its training set).

Without these practices as a standard part of their methodology, data scientists might inadvertently bake automated algorithmic gullibility into their neural networks.

As our lives increasingly rely on AI to do the smart thing in all circumstances, these adversarial vulnerabilities might prove catastrophic. That’s why it’s essential that data scientists and AI developers put in place suitable safeguards to govern how AI apps are developed, training, and managed.

Infoworld

You Might Also Read: 

A Revolution In Warfare Made Possible By AI:

Using AI In Business Intelligence:

« Equifax Executives Resign Without Charge
Kaspersky Says We Can Trust Him »

CyberSecurity Jobsite
Check Point

Directory of Suppliers

Clayden Law

Clayden Law

Clayden Law advise global businesses that buy and sell technology products and services. We are experts in information technology, data privacy and cybersecurity law.

Authentic8

Authentic8

Authentic8 transforms how organizations secure and control the use of the web with Silo, its patented cloud browser.

BackupVault

BackupVault

BackupVault is a leading provider of automatic cloud backup and critical data protection against ransomware, insider attacks and hackers for businesses and organisations worldwide.

LockLizard

LockLizard

Locklizard provides PDF DRM software that protects PDF documents from unauthorized access and misuse. Share and sell documents securely - prevent document leakage, sharing and piracy.

CSI Consulting Services

CSI Consulting Services

Get Advice From The Experts: * Training * Penetration Testing * Data Governance * GDPR Compliance. Connecting you to the best in the business.

Australian Signals Directorate (ASD)

Australian Signals Directorate (ASD)

The Australian Signals Directorate is an intelligence agency in the Australian Government Department of Defence.

Northwave

Northwave

Northwave offers an Intelligent combination of cyber security services to protect your information.

NT Cyfence

NT Cyfence

CAT Cyfence is the IT Security services business unit of CAT Telecoms.

CTERA Networks

CTERA Networks

CTERA provides cloud storage solutions that enable service providers and enterprises to launch managed storage, backup, file sharing and mobile collaboration services using a single platform.

Spanish Network of Excellence on Cybersecurity Research (RENIC)

Spanish Network of Excellence on Cybersecurity Research (RENIC)

RENIC is a membership based sectoral association that includes research centers and other agents of the research cybersecurity ecosystem in Spain.

IoT Defense

IoT Defense

IoT Defense (IOTD) is a cybersecurity and networking company building solutions that enable the protection of networks and the ever-increasing prevalence of IoT devices.

Elitecyber Group

Elitecyber Group

Elitecyber group is a team of Cyber Security recruitment experts who work for Cyber Security and Cyber Defence clients and candidates throughout Europe.

ioXt Alliance

ioXt Alliance

The ioXt Alliance is a group of manufacturers, industry alliances and government organizations dedicated to harmonizing best security practices in a highly connected world.

StrataCore

StrataCore

StrataCore is a single-source technology lifecycle advocate that works behind IT teams as a strategic partner to help them achieve peak enterprise outcomes.

Zuul IoT

Zuul IoT

Zuul take an asset-centric approach to OT security, enabling security teams to protect the critical IIoT/IoT devices that are at the foundation of critical business functions.

Fullstack Academy

Fullstack Academy

A trailblazer in bootcamp education, Fullstack Academy prepares students for fulfilling careers in tech through our NYC campus, online learning, and university partnerships.

Locuz

Locuz

At Locuz, we’ve made it our mission to help businesses like yours create an actionable digital strategy.

Entro Security

Entro Security

Entro is the first holistic secrets security platform that detects, safeguards, and enriches with context your secrets across code, vaults, chats, and platforms.

SafeLiShare

SafeLiShare

SafeLiShare’s data security platform unifies encryption strategies for organizations with hybrid and multi-cloud infrastructures, ensuring data is secure regardless of its location.

CloudBees

CloudBees

CloudBees is building the world’s first end-to-end automated software delivery system, enabling companies to balance governance and developer freedom.

Xeliumtech Solutions

Xeliumtech Solutions

Xeliumtech Solutions are a Digital Transformation partner with quality offerings in Mobile App Development, Ecommerce, Devops, RPA, AI, IoT development, Cybersecurity and more.