A much-discussed MIT study suggested that more than 95% of artificial intelligence initiatives never make it to production or deliver measurable ROI. In healthcare, the failure rate may be even higher, according to Michael Privat, chief data and engineering officer of Availity, an IT services and consulting firm.
The Operationalization Problem
Privat notes a consistent pattern: “Teams build prototypes quickly and generate excitement, but the moment they try to operationalize those solutions, everything changes. Now you’re dealing with real patient data, regulatory requirements, governance and security risks.” Most companies attempting to operationalize health AI skip critical foundational work, instead simply buying a model, plugging it into a notebook, and calling it a pilot. True operationalization requires building data lineage and identity resolution at production scale, audit trails that satisfy HIPAA and SOC 2, and a continuous evaluation framework. Model drift is a significant challenge—a model that works in January may stop working by July without proper monitoring.
Common Pitfalls and Success Factors
A primary pitfall is using AI as an additive layer on top of already-broken processes. “AI amplifies what exists,” Privat explains. If a prior authorization workflow has 14 handoffs and a two-week queue, generative AI will simply produce more material for the same bottleneck. Other mistakes include scoping pilots to demonstrate capability rather than deliver results, and allowing vendors to define success metrics. Lasting initiatives require what Privat calls an “honest owner”—someone on the business or clinical side who must defend outcomes—and a real metric tied to profit and loss or patient outcomes, not engagement or queries served.
Scaling and Observability
Real organizational scaling requires consolidation: one inference platform, one evaluation framework, one identity and data model, and one set of guardrails. Standardization, not novelty, is what enables AI to scale cost-effectively. Governance should function as infrastructure through capability gates and audit trails, not as a committee creating approval bottlenecks. End-to-end observability differs from simple monitoring. Organizations need input drift detection, output quality signals tied to continuous evaluation, latency and cost tracking, and full audit trails. Healthcare regulators will require reproducible explanations for specific case handling; “the model said so” is insufficient. Observability requires pairing every model with downstream metrics that break when the model breaks, ensuring organizations monitor outcomes rather than just system uptime.
This article is an AI-assisted summary. All facts and figures are drawn from the original report: https://www.healthcareitnews.com/news/qa-why-pricey-ai-prototypes-are-often-left-cutting-room-floor