THE PROBLEM
Why this system existed
Rules-based triage consumed analyst time and struggled with document variation, while ambiguous cases still required review.
BERT · Classification · MLflow
Mortgage Document Classification
A model-first document workflow replacing brittle rules while preserving human oversight of low-confidence predictions.
MY ROLETransformer modeling, confidence routing, evaluation, and API delivery
SCOPEA confidence-gated BERT classification pipeline for mortgage-originations document categories with human review for ambiguous cases.
EVIDENCEProfessional experience · public-safe summary
93.4%macro F1
8.2 → 1.4hours per batch
12document categories
OUTCOME
What changed
Reached 93.4% macro F1 and reduced analyst time per batch from 8.2 hours to 1.4 hours.
REFERENCE ARCHITECTURE
Controls around the model
01Classify
02Route
03Retrieve
04Rerank
05Generate
06Validate
07Observe
DECISIONS
Trade-offs considered
Confidence-gated automation
Human review tail
Tracked model variants
Compliance-approved REST deployment
FAILURE CASE
What did not work
Automation without confidence routing would hide uncertainty; below-threshold documents remained in the analyst queue.
SECURITY BOUNDARY
Public-safe by design
This case study exposes patterns, not employer architecture. It uses synthetic data, no client identifiers, no internal prompts, no proprietary datasets, and no production endpoints.