THE PROBLEM
Why this system existed
High-volume domain tasks need strong quality and lower unit cost without sacrificing reproducibility or rollback.
Fine-tuning · MLOps · Cost Control
Domain-Adapted Llama 3
QLoRA fine-tuning, MLflow experiment tracking, vLLM serving, and staged promotion with quality checks at each gate.
MY ROLEDataset curation, QLoRA fine-tuning, evaluation, serving, and rollback
SCOPEA domain-adapted model workflow for classification and standards-lookup tasks with reproducible training, staged promotion, and evaluation gates.
EVIDENCEProfessional experience · public-safe summary
0.74 → 0.91task F1
38%lower cost
34tracked runs
OUTCOME
What changed
Improved task F1 from 0.74 to 0.91 and reduced per-query inference cost by 38%.
REFERENCE ARCHITECTURE
Controls around the model
01Classify
02Route
03Retrieve
04Rerank
05Generate
06Validate
07Observe
DECISIONS
Trade-offs considered
Parameter-efficient QLoRA adaptation
Dataset and hyperparameter versioning
Canary promotion
Evaluation-gated rollback
FAILURE CASE
What did not work
Fine-tuning gains are task-specific; model promotion stayed gated by domain evaluation rather than benchmark performance alone.
SECURITY BOUNDARY
Public-safe by design
This case study exposes patterns, not employer architecture. It uses synthetic data, no client identifiers, no internal prompts, no proprietary datasets, and no production endpoints.