Back to site

Programme analytics

What the agent is doing to the queue, and whether the humans who check it still agree.

Alerts investigated

60,504

trailing 12 weeks

Auto-close rate

86%

from 52% at week 1

Median investigation

2m 35s

intake to released narrative

Cost avoided

$1,250,926

against a fully staffed L1/L2 function

Weekly throughputAuto-closedRouted to a human
04K7KW1W1: 3,100 alerts, 1,488 routed to a humanW2W2: 3,538 alerts, 1,653 routed to a humanW3W3: 3,961 alerts, 1,784 routed to a humanW4W4: 4,358 alerts, 1,870 routed to a humanW5W5: 4,720 alerts, 1,901 routed to a humanW6W6: 5,044 alerts, 1,876 routed to a humanW7W7: 5,333 alerts, 1,797 routed to a humanW8W8: 5,593 alerts, 1,669 routed to a humanW9W9: 5,837 alerts, 1,507 routed to a humanW10W10: 6,078 alerts, 1,322 routed to a humanW11W11: 6,332 alerts, 1,132 routed to a humanW126.6KW12: 6,610 alerts, 946 routed to a human

12 weeks · auto-close share rises as typologies are enabled

QA concurrence — reviewer agreement with the agent
0.900.951.00W1W1: 92.1% concurrenceW2: 92.8% concurrenceW3W3: 93.5% concurrenceW4: 94.1% concurrenceW5W5: 94.7% concurrenceW6: 95.2% concurrenceW7W7: 95.7% concurrenceW8: 96.1% concurrenceW9W9: 96.5% concurrenceW10: 96.9% concurrenceW11W11: 97.3% concurrenceW12: 97.8% concurrence

Target floor 0.95 · sampled weekly

Disposition mix
  • Cleared3 · 60%
  • Escalate to L32 · 40%

Status colours are reserved and always carry a label

Calibration — predicted confidence vs observed agreement
0.40.40.60.60.80.81.01.0Predicted 0.45 → observed 0.44 (n=612)Predicted 0.55 → observed 0.57 (n=1,840)Predicted 0.65 → observed 0.63 (n=3,910)Predicted 0.75 → observed 0.77 (n=7,204)Predicted 0.85 → observed 0.84 (n=12,460)Predicted 0.95 → observed 0.96 (n=15,174)

Perfect calibration tracks the diagonal

Reading these numbers

Auto-close rate is the number a CFO buys on, but it is not the number that keeps the programme safe. The pair to watch is auto-close against QA concurrence: a rising auto-close rate with falling concurrence means the agent is clearing cases a human would not have cleared, and the control band should pull the affected segment back to manual routing before the trend reaches an examination.

The calibration curve is the honesty check. Confidence is only useful if it means something — a stated 75% has to be right about 75% of the time, or the routing threshold that sits on top of it is arbitrary.

Escalation volume is deliberately not minimised. Under-reporting and over-reporting are both penalised, so the target is a stable escalation rate with high precision, not the lowest possible number of referrals.