The Security Risks Behind Shadow ML Adoption

Uploaded on 2025-04-07 in TECHNOLOGY-Key Areas-Artificial Intelligence, TECHNOLOGY--Resilience, FREE TO VIEW

Artificial Intelligence (AI) is at the centre of a global technological arms race, with enterprises and governments pushing the boundaries of what’s possible. The launch of DeepSeek has once again ignited discussions around AI’s sophistication and cost of development. However, as AI models become more advanced and widely deployed, security concerns continue to mount.

Companies rushing to keep pace with developments like DeepSeek risk cutting corners, leaving vulnerabilities that adversaries can exploit.

A key concern is the rise of “Shadow ML”, where machine learning models are deployed without IT oversight, bypassing security protocols, compliance frameworks, and data governance policies. This proliferation of unauthorised AI tools introduces a host of security risks, from plagiarism and model bias to adversarial attacks and data poisoning. If left unchecked, these risks can undermine the integrity and trustworthiness of AI-driven decisions in critical sectors like finance, healthcare, and national security.

Software Is Critical Infrastructure

Software is now a central component of modern infrastructure, akin to electricity grids and transportation networks. Failures in these systems can cascade across industries, causing widespread disruption. With AI/ML models now embedded in core software operations, the potential impact of security breaches is even more severe.

Unlike traditional software, AI models operate more dynamically and unpredictably. They can continuously learn and adapt based on new data, meaning their behaviour can change over time—sometimes in unintended ways. Attackers can exploit these evolving behaviours, manipulating models to generate misleading or harmful outputs. The growing reliance on AI-driven automation makes it imperative to establish robust MLOps security practices to mitigate these emerging threats.

The Security Challenges In MLOps

The AI/ML model lifecycle presents several key vulnerabilities. One of the primary concerns is model backdooring, where pre-trained models can be compromised to produce biased or incorrect predictions, affecting everything from financial transactions to medical diagnoses. Data poisoning is another major risk, as attackers can inject malicious data during training, subtly altering a model’s behaviour in ways that are difficult to detect.

Additionally, adversarial attacks - where small modifications in input data trick AI models into making incorrect decisions - pose a serious challenge, particularly in security-sensitive applications.

Later in the lifecycle, implementation vulnerabilities also play a critical role in AI security. Weak access controls can lead to authentication gaps, allowing unauthorised users to tamper with models or extract sensitive data. Improperly configured containers that host AI models can provide an entry point for attackers to access broader IT environments. Moreover, the use of open-source ML models and third-party datasets increases supply chain risks, making it critical to verify the integrity of every component.

While AI promises groundbreaking advancements, security cannot be an afterthought. Securing AI can make the technology even more appealing for businesses. Organisations must prioritise secure MLOps practices to prevent cyber threats from exploiting the very tools designed to enhance corporate efficiency and decision-making.

Best Practices For Secure MLOps

To defend against evolving threats targeting AI models, organisations should adopt a proactive security posture. Model validation is key to identify potential biases, malicious models, and adversarial weaknesses before deployment. Dependency management ensures that ML frameworks and libraries- like TensorFlow and PyTorch- are sourced from trusted repositories and scanned for malicious model threats. Code security should also be a priority, with static and dynamic analysis conducted on source code to detect potential security flaws in AI model implementations. However, security shouldn’t stop at the source code level - threats can also be embedded within compiled binaries. A comprehensive approach must include binary code analysis to detect hidden risks, like supply chain attacks, malware, or vulnerable dependencies that may not be visible in the source code.

On top of securing AI code, organisations must harden container environments by enforcing strict policies on container images, ensuring they are free from malware and misconfigurations. Digitally signing AI models and related artifacts helps maintain integrity and traceability throughout the development lifecycle. Continuous monitoring should also be implemented to detect suspicious activity, unauthorised access, or unexpected deviations in model behaviour. By embedding these security measures into the AI development lifecycle, companies can create resilient MLOps pipelines that balance innovation with robust protection.

The Future Of AI Security

As AI adoption accelerates, the conflict between innovation and security will intensify. AI is not just another tool, it's a critical asset that needs dedicated security strategies. The rise of Agentic AI, with its ability to make autonomous decisions, adds another layer of complexity, making governance and oversight more important than ever. Organisations that take a proactive approach now are better positioned to navigate these evolving risks without slowing down innovation.

The launch of DeepSeek and similar AI advancements will continue to reshape industries, but the rush to innovate must not come at the expense of security.

Just as we wouldn’t build a skyscraper without a solid foundation, we cannot deploy AI without embedding security into its very core. The organisations that succeed in this new AI-driven world will be those that recognise security as an enhancer, not a barrier, to progress.

By taking a proactive stance on AI security, enterprises can ensure that they are not only keeping up with the latest developments but also safeguarding their future in an increasingly AI-powered world.

Shachar Menashe is VP of Security Research at JFrog

You Might Also Read:

Half of Employees Use Shadow AI:

If you like this website and use the comprehensive 7,000-plus service supplier Directory, you can get unrestricted access, including the exclusive in-depth Directors Report series, by signing up for a Premium Subscription.