/ ENGINEERING SYSTEMS
← Projects

BERT · Classification · MLflow

Mortgage Document Classification

2026BERT

A model-first document workflow replacing brittle rules while preserving human oversight of low-confidence predictions.

THE PROBLEM

Why this system existed

Rules-based triage consumed analyst time and struggled with document variation, while ambiguous cases still required review.

OUTCOME

What changed

Reached 93.4% macro F1 and reduced analyst time per batch from 8.2 hours to 1.4 hours.

REFERENCE ARCHITECTURE

Controls around the model

01Classify
02Route
03Retrieve
04Rerank
05Generate
06Validate
07Observe

DECISIONS

Trade-offs considered

Confidence-gated automation
Human review tail
Tracked model variants
Compliance-approved REST deployment

FAILURE CASE

What did not work

Automation without confidence routing would hide uncertainty; below-threshold documents remained in the analyst queue.