Problem: 70% of tickets fell into 3 common categories. Remaining 30% across 14 specialist queues were effectively randomly routed. Misrouted specialist tickets had 3× longer resolution time. A keyword classifier alone scored 71% overall precision.
Solution: Stage 1: fine-tuned DistilBERT on 18 months of labelled tickets (confidence threshold 0.72). Stage 2: Claude API for sub-threshold tickets — structured prompt with queue descriptions and 2 examples per queue, JSON-constrained output. n8n orchestrated ingest (IMAP, Slack webhook, REST), classification, routing via ticketing API, and metric emission. Precision dashboard from Postgres events table updated every 5 minutes.
Technology: n8n · DistilBERT · Claude API · Postgres · Slack · Python
Optimisation pattern: keyword-classifier-to-two-stage-ml-llm-fallback-with-live-precision-dashboard
Outcomes:
Overall routing precision: 94% (vs 71% baseline). Tail-class precision: 89%. Manual triage: 1.5 FTE → 0.1. Average ticket resolution time reduced 31%.