MLOps Engineer Manager

Setting cloud strategy for ML, evaluating and selecting new services (e.g., Bedrock vs. self-hosted LLMs), governing enterprise-wide IAM and networking policies for ML platforms, and optimising cloud spend.

Defining the enterprise containerisation strategy, making build-vs-buy decisions on orchestration platforms (e.g., EKS vs. OpenShift), and owning platform-level security and governance for ML workloads.

Designing the entire CI/CD for ML framework for the organisation, integrating security (SAST/DAST) and cost management tools into the platform, and managing the enterprise Terraform state and module registry.

Owning the ML platform roadmap, selecting and integrating enterprise-wide tooling (e.g., MLflow vs. Weights & Biases), and designing the meta-orchestration layer connecting data, training, and serving.

Defining the organisation's observability strategy for ML systems, integrating logging, metrics, and tracing across the entire stack, and championing and implementing concepts like MLOps control towers.

• Head of Data Science
• Product Engineering Leads
• Cloud Infrastructure Team
• Cyber Security Lead
• Finance Business Partner

• Cloud Service Providers (e.g., AWS account managers)
• MLOps Tooling Vendors
• Industry Peer Groups

Large Language Models (LLMs) and generative AI are transforming product capabilities and developer workflows. As an MLOps leader, you'll need to understand how to productionise, monitor, and cost-optimise these models, which have unique challenges compared to traditional ML.

Prompt Engineering & Management · Retrieval Augmented Generation (RAG) Architectures · LLM Observability & Evaluation · Fine-tuning & Model Customisation

• This month: Experiment with a few LLM APIs (e.g., OpenAI, Anthropic) to understand their capabilities and limitations. Try building a simple RAG application.
• Next quarter: Task your team with researching and prototyping an LLM-powered internal tool (e.g., a code assistant or documentation generator).
• Month 4-6: Develop a framework for evaluating LLM performance and cost-effectiveness for potential production use cases.
• Month 7-9: Start planning for the infrastructure required to support internal LLM fine-tuning or deployment, considering GPU resources and data pipelines.

Quick win: Encourage your team to use AI coding assistants like GitHub Copilot daily. It's a low-risk way to get familiar with generative AI in a practical context.

As ML systems become more critical, they also become attractive targets for attacks (e.g., data poisoning, model evasion). Integrating security practices throughout the MLOps lifecycle (MLSecOps) is no longer optional; it's essential for protecting our models and data.

Adversarial Attacks & Defences · Secure ML Data Pipelines · Model Vulnerability Scanning · Threat Modelling for ML Systems

• This quarter: Partner with our Cyber Security team to understand their current concerns and how they apply to ML systems. Identify existing security gaps in our MLOps practices.
• Next quarter: Lead a threat modelling exercise for one of our critical production ML models, involving your team and security experts.
• Month 4-6: Research and evaluate tools or frameworks for integrating security scanning and vulnerability detection into our ML CI/CD pipelines.
• Month 7-9: Develop and implement a plan to address the highest-priority security risks identified in your threat modelling exercise.

Quick win: Ensure all ML-related cloud resources adhere to the principle of least privilege. It's a fundamental security practice that's often overlooked in ML contexts.