We ran AI through our own helpdesk for six months before we let it near a client queue. Here is the honest scorecard — two things we kept, two we killed, and one that caught us off guard.
What we kept
triage classification
Every inbound ticket gets a category, a priority, and a suggested runbook before a human opens it. Accuracy sits around 94% on our queue, and a wrong guess costs exactly one click to correct.
draft-first responses
The model writes the first reply, an engineer edits and sends. Nothing goes out unread. Median first-response time went from 41 minutes to 9.
What we killed
autonomous resolution
We let it close low-risk tickets on its own for three weeks. It was right most of the time, and "most of the time" is not a support standard. Humans close tickets.
chatbot as tier one
Users hated it, and they could tell. If somebody is frustrated enough to open a ticket, making them argue with a bot first is a tax, not a service.
The surprise
The biggest win was not on the ticket at all — it was the write-up afterward. Root-cause summaries that engineers used to skip now get drafted straight from the ticket thread, and our runbook library grew more in six months than in the previous two years.
We use AI like any other tool: on the repetitive work, wired into real systems, and never holding the pen on somebody else's outage.
What this means for your queue
If you are evaluating AI for support, start where the cost of being wrong is a keystroke rather than an outage. Classification, drafting, and documentation clear that bar comfortably. Closing tickets does not — at least not yet.