Analytics Blog

Monkey See, Monkey Can’t Do: Why Traditional BI & Analytics Tools are unable to fulfill the promise of Big Data in Healthcare

The healthcare and life science industries have been devoted users of traditional statistical analysis tools and business intelligence reporting systems for decades, and causal analysis in tightly controlled studies has been de rigueur in the increasingly strict legal and regulatory constructs that have emerged over 70 plus years dating back to the 1938 Food Drug and Cosmetic Act.  Tools and techniques for these requirements are well established, well designed, and entrenched in existing drug development and clinical research practice.

However well suited these tools may be for tasks requiring relatively small sample sizes and well defined research protocols, especially when used in studies that can span several years of work, they are simply not up to the demands of advanced predictive analytics that is critically needed to navigate an increasingly complex policy and technology landscape.  The rapidly growing availability, size, and variety of data simply capsize software designed in a time and for a purpose that had no concept of “Big Data”.

Alpine Data Labs was created precisely for the purpose of analyzing vast troves of data and to deliver robust, statistically reliable predictive models from that data. Alpine’s advanced models make it possible to garner meaningful insight and enable better, data based decisions than would otherwise be possible.  We have consciously chosen not to pursue traditional “Small Data” business, a well-served market by incumbents, and instead to focus our intellectual capital, resources and energy to where the puck is headed; large-scale predictive modeling based on statistically significant correlation found in diverse, very large, data sets.

Healthcare’s urgent economic realities, combined with a changing policy landscape, increasingly requires all participants to demonstrate and deliver tangible results in the real world, with outcomes based on real world evidence, in order to be paid.  The shift from traditional fee for health service rendered and life science product purchased is grindingly, but unavoidably, moving towards a pay-for-performance, outcomes-based payment model.

Outcomes based performance questions cannot be answered by trying to adapt or accommodate small data software solutions in a way that would solve Big Data problems.  For example, while a five-year study might be sufficient and necessary to prove a drug is safe and effective for human use, no one has the luxury of that many years to decide on an actionable course of treatment for a population of patients that not only share a particular disease or condition, but who also share common economic and environmental factors, comorbidities, and behavioral patterns that can potentially be correlated to the success or failure of a particular treatment decision.  To quote the CEO of Infochimps, Jim Kaskade, the “unexperienced and unobservable” can be uncovered from “real experiences and real observations”.  For example, conducting a randomized controlled study with a cohort of smokers who must continue to smoke as part of a smoking cessation study to maintain the study’s objectivity creates an ethical dilemma that Big Data modeling of real world smoking behavior simply does not.

That’s a Big Data question, and it takes Big Data thinking and capabilities.

Vendors of traditional statistical analysis, data warehouse, and business intelligence software simply are incapable of quickly or easily adapting their existing products and business models to the scale of Big Data, at least not without making major sacrifices and tradeoffs that impact their primary revenue generating products and services.  In other words, even though these vendors can clearly see the need and growth of Big Data market requirements, it doesn’t mean they are ready to address the hurdles Big Data represents for their product offering, and more importantly, how to address the inhibitors and risks they create for their customers when they try to convince them otherwise.

Monkey see, but Monkey can’t do.