By now, most workers’ compensation organizations have run an AI pilot. Some have run several. A meaningful number of those pilots produced results that looked promising — faster processing, better triage accuracy, fewer touches per claim. And then… nothing. The pilot ended. A report got filed. Things went back to normal.
This is not a small problem. Only 30% of carriers have achieved desired benefits from their AI investments to date. That means 70% of insurers that have invested real money and real time into AI are not seeing the returns they expected. If that number surprised you, it probably shouldn’t.
The technology isn’t the issue. What’s happening is something more organizational, and more fixable.
Recognizing Pilot Purgatory
Pilot purgatory is what happens when an AI project succeeds on its own terms but fails to survive contact with the real operation. The pilot worked in a controlled environment, with clean data, a dedicated project team, and management attention. When it ended, nobody had figured out how to integrate it into the actual workflow, who was responsible for maintaining it, or how to measure its ongoing impact.
So, it sits. Or it gets quietly discontinued. Or it becomes a slide in a conference presentation about innovation while the underlying operation continues running the same way it always has.
Sound familiar? You’re not alone. This pattern is consistent enough across the industry that it has a name. And the carriers, TPAs, and self-insured groups that have escaped it tend to share a few things in common.
Why It Keeps Happening
A few dynamics consistently produce this outcome:
Pilots are designed to succeed in isolation. When the goal is to prove a concept, you design for that. Clean data, a narrow use case, a team that’s fully focused on making it work. That’s not the real operating environment, and results that hold in a pilot don’t automatically transfer.
Integration complexity gets underestimated. The moment you try to connect an AI tool to a live claims system, payroll integration, medical bill review workflow, or reporting infrastructure, you hit a different category of problem. Insurers that haven’t invested in modern, API-driven platforms often find that the last 20% of an AI implementation (read: connecting it to everything else) costs more than the first 80%.
Ownership is unclear. Who runs this after the pilot team disbands? If that question doesn’t have a clear answer before the project starts, the answer after it ends is usually nobody.
Success metrics don’t survive the transition. Pilot metrics tend to focus on what’s easy to measure in a controlled environment. When the project goes live, nobody has defined what success looks like in the actual operation, so it’s easy to declare it inconclusive and move on.
What the 30% Do Differently
The insurers seeing real returns from AI treat AI as an operational capability to be built into the workflow, not a technology to be evaluated and then shelved.
That means starting with the workflow, not the technology. What specific decision or process is this improving — claims triage, medical bill review, intake routing? Who uses it? What does their day look like before and after? What does success mean six months from now?
It means building on infrastructure that can support integration, cloud-native platforms with open APIs that allow AI tools to connect to live data rather than working from static exports.
And it means identifying an internal owner before the pilot ends. This critical role involves accountability for performance, adoption, and continuous improvement after go-live.
The gap between those who are getting results from AI and those who aren’t is rarely about the technology. It’s about whether the organization was ready to absorb it. The good news is that readiness is a choice, not a given.
If your insurance organization has completed an AI pilot that never fully translated into operational impact or you’re just not sure where to start, connect with Ryan Smith, Senior Solutions Advisor. He can discuss how True helps insurers move AI out of the proof-of-concept stage and into live workflows where it produces measurable results.
