/ ENGINEERING SYSTEMS
← Projects

Fine-tuning · MLOps · Cost Control

Domain-Adapted Llama 3

2026Fine-tuning

QLoRA fine-tuning, MLflow experiment tracking, vLLM serving, and staged promotion with quality checks at each gate.

THE PROBLEM

Why this system existed

High-volume domain tasks need strong quality and lower unit cost without sacrificing reproducibility or rollback.

OUTCOME

What changed

Improved task F1 from 0.74 to 0.91 and reduced per-query inference cost by 38%.

REFERENCE ARCHITECTURE

Controls around the model

01Classify
02Route
03Retrieve
04Rerank
05Generate
06Validate
07Observe

DECISIONS

Trade-offs considered

Parameter-efficient QLoRA adaptation
Dataset and hyperparameter versioning
Canary promotion
Evaluation-gated rollback

FAILURE CASE

What did not work

Fine-tuning gains are task-specific; model promotion stayed gated by domain evaluation rather than benchmark performance alone.