I spent 18 years automating software delivery. When I started moving into ML infrastructure, my first instinct was to map everything back to what I already knew: pipelines, artifacts, environments, monitoring. That works — up to a point. Then you hit the things that genuinely don't map, and those are the interesting bits.

Pipeline Stages at a Glance

DevOps Code → Build → Test → Deploy → Operate

MLOps Data → Features → Train → Evaluate → Register → Serve → Monitor

LLMOps Select → Optimise → Serve → Evaluate → Monitor → Iterate

DevOps: The Deterministic Baseline

DevOps is built on a comfortable assumption: your artifact is code, and code is deterministic. If the tests pass today and you haven't changed anything, they'll pass tomorrow. The feedback loop is binary — it works or it doesn't. This is why CI/CD tooling is so powerful here. Jenkins, GitHub Actions, ArgoCD — they all work because the rules of the game are stable.

Artifact: compiled code, container image, IaC plan
Versioning: Git — a single source of truth
Monitoring: uptime, error rates, latency, CPU/memory — all objective
Failure mode: something broke; the logs tell you what

The DevOps mental model works because software doesn't degrade on its own. A deployed binary doesn't get worse at its job between Monday and Friday unless someone changes it. That assumption quietly breaks in ML.

MLOps: When the World Changes Under Your Feet

The first time I heard the phrase "model drift," I assumed it was a fancy way of saying "the model has a bug." It's not. A model can degrade in production with zero code changes — because the data distribution shifted, the user behaviour changed, or the business context evolved. The code is fine. The model is lying to you.

This is the core insight that forces MLOps to exist as a separate discipline. You can't solve a data problem with a code pipeline.

Artifact: a trained model, feature set, and its training data — all three must be versioned together
Versioning: DVC for data and models alongside Git for code; experiment tracking in MLflow
Monitoring: accuracy, precision, recall, data drift (statistical tests on input distributions), concept drift
Failure mode: silent — the model keeps returning results, they're just quietly wrong

The tooling shift is real. Kubeflow Pipelines or Apache Airflow orchestrates training workflows where Jenkins used to orchestrate builds. MLflow tracks experiments the way Git tracks commits. Amazon SageMaker bundles a lot of this for AWS shops. The concept of a pipeline is familiar; the artefacts it manages are not.

Where DevOps knowledge transfers

Containerisation — ML workloads run in Docker/OCI containers on Kubernetes, same as everything else
Infrastructure-as-Code — SageMaker training jobs, feature store configs, endpoint definitions all live in Terraform or CDK
Observability thinking — you already know how to instrument, alert, and page; the metrics just change
GitOps — model promotion flows (staging → prod) map well onto GitOps patterns with ArgoCD or Flux

Where you hit a wall

Data versioning — there is no git diff for a 500 GB Parquet dataset; you need DVC or LakeFS
Experiment reproducibility — re-running a training job isn't like re-running a build; seeds, framework versions, and data snapshots all matter
Non-determinism — two identical training runs can produce models with meaningfully different behaviour
Evaluation gates — "does it work?" requires a held-out test set, business metrics, and often human review; you can't just check exit codes

LLMOps: Foundation Models Break the Last Assumptions

LLMOps is where things get genuinely strange for a systems engineer. With classical ML, you at least own the model — you trained it, you know its architecture, you can retrain it. With LLMs, most of the time you're working with a foundation model someone else trained. You're tuning inputs, not weights. Your primary levers are prompt engineering, retrieval-augmented generation (RAG), and fine-tuning — in roughly that order of cost and complexity.

Artifact: a prompt template, a RAG index, a fine-tuned adapter (LoRA), or some combination
Versioning: prompt versions in a prompt registry; vector index snapshots; model versions in a registry
Monitoring: hallucination rate, groundedness, bias/toxicity, token costs, latency-per-token, human feedback scores
Failure mode: the model confidently returns something wrong, offensive, or wildly off-topic — and it's hard to write a unit test for that

In DevOps, a failed test tells you: fix the code, re-run the test. In MLOps, a drop in accuracy tells you: retrain with more/better data. In LLMOps, a failed eval might mean any of: your prompt is under-specified, your RAG retrieval is returning irrelevant context, the base model isn't suited for this task, or you need fine-tuning. You can't always tell which lever to pull from the eval result alone.

Concrete tools at the LLMOps layer

Serving: vLLM, TGI (Text Generation Inference), NVIDIA Triton — these are to LLMs what gunicorn is to Flask
Observability: LangSmith, Helicone (proxy-based tracing), Phoenix (Arize) — trace every prompt/response pair, flag regressions
Evaluation: RAGAS for RAG pipeline quality, DeepEval, human-in-the-loop review workflows
Cost control: token budgeting, model routing (GPT-4o for complex queries, smaller model for classification) — this is FinOps for inference

At a Glance

DevOps

Artifact Code, container image, IaC plan

Failure mode Explicit — tests fail, services crash

Versioning Git

Monitoring Uptime, latency, error rate

Key tools GitHub Actions, ArgoCD, Terraform

MLOps

Artifact Model + training data + hyperparams

Failure mode Silent — accuracy degrades over time

Versioning Git + DVC + MLflow

Monitoring F1, data drift, concept drift

Key tools Kubeflow, MLflow, SageMaker

LLMOps

Artifact Prompt + RAG index + LoRA adapter

Failure mode Qualitative — hallucination, bias

Versioning Prompt registry, index snapshots

Monitoring Groundedness, token cost, RLHF scores

Key tools LangSmith, Phoenix, vLLM, RAGAS

The monitoring granularity shifts from objective (CPU%) → statistical (F1 score) → qualitative (does this answer make sense?). That last step is the one that still makes me uncomfortable as a systems engineer — there's no clean alert threshold for "hallucinating too much."

What This Actually Means If You're Coming from DevOps

The good news: your infrastructure instincts are directly valuable. Kubernetes, GitOps, observability, IaC, cost management — all of it applies. The ML and LLM layers still run on the same compute fabric you've been managing for years.

The honest news: the artefacts are different, the failure modes are different, and some of the evaluation is fundamentally subjective. You need to build comfort with probabilistic thinking and qualitative assessment — two things that don't come naturally when you've spent two decades chasing deterministic systems.

My practical advice for the transition:

Get hands-on with MLflow tracking first — it makes the experiment lifecycle concrete and maps well onto things you already understand
Run a RAG pipeline end-to-end before trying to optimise anything — understand what a vector index actually retrieves before you tune prompts
Treat token costs like cloud compute costs — model them early, set budgets, alert on anomalies
Accept that "it works" now has a confidence interval, not a boolean value

Key Takeaways

DevOps handles deterministic code artefacts; MLOps handles models that decay; LLMOps handles foundation models you didn't train
Data versioning (DVC, LakeFS) and experiment tracking (MLflow) are the first new skills to acquire in MLOps
LLMOps evaluation is non-linear — a failed eval has multiple possible root causes across prompt, retrieval, and model layers
Observability tools like LangSmith and Phoenix are essential; you cannot operate LLMs blind
The infrastructure muscle memory (Kubernetes, GitOps, IaC) transfers directly — the artefact management and evaluation mindset do not

Articles & Posts

DevOps vs MLOps vs LLMOps