The Seven Pillars Of MLops

To unlock the full potential of AI and Machine Learning, organisations must focus on model selection, optimisation, monitoring, scaling, and metrics for success.

Integrating AI and machine learning into business operations is no longer just for companies aiming to stay ahead in an ever-evolving technological landscape. However, many organisations still face challenges in tapping into AI/ML’s true power.

To help resolve this, I’ve explored key trends in MLops and compiled actionable insights to overcome common engineering hurdles.

As you might expect, generative AI models differ significantly from traditional machine learning models in their development, deployment, and operational requirements. I’ll walk through these differences, which range from training and the delivery pipeline to monitoring, scaling, and measuring model success, and leave you with a few key questions organisations should address to guide their AI/ML strategy.

Ultimately, by focusing on solutions, not just models, and by aligning MLops with IT and DevOps systems, organisations can unlock the full potential of their AI initiatives and drive measurable business impacts.

Laying The Groundwork For MLops Success

Like many things in life, to successfully integrate and manage AI and ML into business operations, organisations first need to have a clear understanding of the foundations. The first fundamental of MLops today is understanding the differences between generative AI models and traditional ML models.

Cost is another major differentiator. The calculations of generative AI models are more complex, resulting in higher latency, demand for more computer power, and higher operational expenses. Traditional models, on the other hand, often utilise pre-trained architectures or lightweight training processes, making them more affordable for many organisations. Recent JFrog data found that 14% of UK organisations are still hesitant to use ML models, with cost and complexity key factors.  When determining whether to utilise a generative AI model versus a standard model, organisations must evaluate these criteria and how they apply to their individual use cases.

Optimising & Monitoring Models Effectively 

Optimising models for specific use cases is crucial. For traditional ML, fine-tuning pre-trained models or training from scratch are common strategies. GenAI introduces additional options, such as retrieval-augmented generation (RAG), which allows the use of private data to provide context and ultimately improve model outputs. Choosing between general-purpose and task-specific models also plays a critical role. Do you really need a general-purpose model or can you use a smaller model that is trained for your specific use case? General-purpose models are versatile but often less efficient than smaller, specialised models built for specific tasks.

Model monitoring also requires distinctly different approaches for generative AI and traditional models. Traditional models rely on well-defined metrics like accuracy, precision, and an F1 score, which are straightforward to evaluate. In contrast, generative AI models often involve metrics that are a bit more subjective, such as user engagement or relevance. Good metrics for genAI models are still lacking and it really comes down to the individual use case. Assessing a model is very complicated and can sometimes require additional support from business metrics to understand if the model is acting according to plan.

However, recent JFrog data found that nearly half (47%) of UK organisations are skipping scans, highlighting just how often this crucial aspect of the model is overlooked. In any scenario, businesses must design architectures that can be measured to make sure they deliver the desired output.

Advances In ML Wngineering Tools 

Traditional machine learning has long relied on open source solutions, from open source architectures like LSTM (long short-term memory) and YOLO (you only look once), to open source libraries like XGBoost and Scikit-learn. These solutions have become the standards for most challenges thanks to being accessible and versatile. For genAI, however, commercial solutions like OpenAI’s GPT models and Google’s Gemini currently dominate due to high costs and intricate training complexities. Building these models from scratch means massive data requirements, intricate training, and significant costs.

Despite the popularity of commercial generative AI models, open-source alternatives are gaining traction. Models like Llama and Stable Diffusion are closing the performance gap, offering cost-effective solutions for organisations willing to fine-tune or train them using their specific data. Interestingly, the UK shows greater caution here, 38% of organisations restrict public software downloads, 10% above the global average, which may limit the ability to fully leverage open source options. Open-source models can present licensing restrictions and integration challenges but overly restrictive policies may further complicate efforts to ensure ongoing compliance and efficiency.  

Scaling ML Systems With Precision

As more and more companies decide to invest in AI, there are best practices for data management and classification and architectural approaches that should be considered for scaling ML systems and ensuring high performance.

Leveraging internal data with RAG

Important questions revolve around data: What is my internal data? How can I use it? Can I train based on this data with the correct structure? One powerful strategy for scaling ML systems with genAI is Retrieval-Augmented Generation. RAG is the ability to use internal data to change the context of a general purpose model. By embedding and querying internal data, organisations can provide context-specific answers and improve the relevance of genAI outputs. For instance, uploading product documentation to a vector database allows a model to deliver precise, context-aware responses to user queries. 

Architectural Considerations For Scalable Systems

Creating scalable and efficient MLops architectures requires careful attention to components like embeddings, prompts, and vector stores. Fine-tuning models for specific languages, geographies, or use cases ensures tailored performance. An MLops architecture that supports fine-tuning is more complicated and organisations should prioritise A/B testing across various building blocks to optimise outcomes and refine their solutions.

Measuring Success Through Meaningful Metrics 

Aligning model outcomes with business objectives is essential. Metrics like customer satisfaction and click-through rates can measure real-world impact, helping organisations understand whether their models deliver meaningful results. Human feedback is essential for evaluating generative models and remains the best practice. Human-in-the-loop systems help fine-tune metrics, check performance, and ensure models meet business goals.

In some cases, advanced generative AI tools can assist or replace human reviewers, making the process faster and more efficient. By closing the feedback loop and connecting predictions to user actions, there is opportunity for continuous improvement and more reliable performance.

Building Solutions, Not Just Models

The success of MLops hinges on building holistic solutions rather than isolated models. Solution architectures should combine a variety of ML approaches, including rule-based systems, embeddings, traditional models, and generative AI, to create robust and adaptable frameworks.

Organisations should ask themselves a few key questions to guide their AI/ML strategies:  

  • Do we need a general-purpose solution or a specialised model?
  • How will we measure success and which metrics align with our goals?
  • What are the trade-offs between commercial and open-source solutions, and how do licensing and integration affect our choices?

Ultimately, the takeaway is clear: AI and ML are not just about creating models, they’re about constructing integrated solutions. These solutions combine multiple components, each influencing the overall user experience and the performance metrics derived from them.

As MLops continues to evolve, organisations must prioritise building scalable, metrics driven architectures. By combining the right tools and strategies, businesses can fully unlock the potential of AI and machine learning, fostering innovation and delivering measurable business outcomes. 

Yuval Fernbach is VP, CTO MLops at JFrog

You Might Also Read: 

The Security Risks Behind Shadow ML Adoption:


If you like this website and use the comprehensive 6,500-plus service supplier Directory, you can get unrestricted access, including the exclusive in-depth Directors Report series, by signing up for a Premium Subscription.

  • Individual £5 per month or £50 per year. Sign Up
  • Multi-User, Corporate & Library Accounts Available on Request

Cyber Security Intelligence: Captured Organised & Accessible


 

 

« TikTok Fined €530M For Breaking EU Rules
Iranian Hackers Attacking Critical Infrastructure »

Infosecurity Europe
CyberSecurity Jobsite
Perimeter 81

Directory of Suppliers

Infosecurity Europe, 3-5 June 2025, ExCel London

Infosecurity Europe, 3-5 June 2025, ExCel London

This year, Infosecurity Europe marks 30 years of bringing the global cybersecurity community together to further our joint mission of Building a Safer Cyber World.

Directory of Cyber Security Suppliers

Directory of Cyber Security Suppliers

Our Supplier Directory lists 8,000+ specialist cyber security service providers in 128 countries worldwide. IS YOUR ORGANISATION LISTED?

NordLayer

NordLayer

NordLayer is an adaptive network access security solution for modern businesses — from the world’s most trusted cybersecurity brand, Nord Security. 

ZenGRC

ZenGRC

ZenGRC (formerly Reciprocity) is a leader in the GRC SaaS landscape, offering robust and intuitive products designed to make compliance straightforward and efficient.

DigitalStakeout

DigitalStakeout

DigitalStakeout enables cyber security professionals to reduce cyber risk to their organization with proactive security solutions, providing immediate improvement in security posture and ROI.

Cyber adAPT

Cyber adAPT

Cyber adAPT offers a leading network threat detection platform (NTD) to the enterprise and ODM/OEM markets.

LexisNexis Risk Solutions

LexisNexis Risk Solutions

LexisNexis Risk Solutions provides technology solutions for Anti-Money Laundering, Fraud Mitigation, Anti-Bribery and Corruption, Identity Management, Tracing and Investigation.

Deltagon

Deltagon

Deltagon develops information security solutions to protect companies’ confidential information in e-communication and e-services.

Ovarro

Ovarro

Ovarro is the new name for Servelec Technologies and Primayer. Ovarro's technology is used throughout the world to monitor, control and manage critical and national infrastructure.

Cloudmark

Cloudmark

Cloudmark is a trusted leader in intelligent threat protection against known and future attacks, safeguarding 12 percent of the world’s inboxes from wide-scale and targeted email threats.

Asvin

Asvin

Asvin provides secure update management and delivery for Internet of Things - IoT Edge devices.

Portshift

Portshift

Portshift leverages the power of Kubernetes and Service-Mesh to deliver a single source of truth for containers and cloud-native applications security.

PNGCERT

PNGCERT

PNGCERT is the national Computer Emergency Response Team (CERT) for Papua New Guinea.

ramsac

ramsac

ramsac provide secure, resilient IT management, cybersecurity, 24 hour support and IT strategy to businesses in London and the South East.

Aiden Technologies

Aiden Technologies

Aiden simplifies your IT process, giving you peace of mind and security by ensuring your computers get exactly the software they need and nothing else.

Axiata Digital Labs

Axiata Digital Labs

Axiata Digital Labs is the technology hub of Axiata Group Berhad Malaysia which is one of the leading groups in telecommunication in Asia.

Custom Computer Specialist (CCS)

Custom Computer Specialist (CCS)

CCS offers an extensive range of services including cybersecurity solutions, consulting, implementation, and support to help our clients maximize the value derived from IT investments.

Holiseum

Holiseum

Holiseum delivers innovative cybersecurity solutions for the critical infrastructure organizations, as well as cybersecurity services and consulting.

TDi Technologies

TDi Technologies

TDI Technologies' flagship solution ConsoleWorks, is an IT/OT cybersecurity and operations platform for Privileged Access Users.

DigiGlass

DigiGlass

DigiGlass is a cutting-edge cybersecurity service provider powered by Redington Gulf, a leading technology distributor in the Middle East and Africa.

InstaSecure

InstaSecure

InstaSecure’s Preventive Cloud Controls accelerate alert remediation and strengthen cloud configurations. Set your controls once and prevent current and future risks.