HDG #021: The many predictive risk models in healthcare
Read time: 5 minutes
Greetings Gurus!
With all the talk about AI lately, I was inspired to take a deeper dive into something that we have taken for granted in healthcare for quite a while now: predictive risk models.
While many, many (many) exist, they are not quite widely or consistently utilized across all organizations. In theory, they could be very powerful tools to foresee or be “tipped off” about potential health outcomes or adverse events waiting to happen, but as with any tool designed to help inform decisions, they come with their own unique set of advantages and limitations.
Predictive Risk Models: A Brief Background
In healthcare, predictive risk models are data-driven algorithms that estimate the likelihood of future health events, such as hospital readmissions or high healthcare costs. These models help clinicians, health systems, and payers identify patients needing additional support or intervention. By leveraging data, we can proactively address health needs and, ultimately, improve outcomes.
By illuminating potential risks and costs, they’re meant to guide healthcare professionals to strategize their intervention approaches, prioritize outreach to high-risk and rising-risk patients or health plan members, and give a “head’s up” about potentially major issues looming on the horizon.
Predictive risk models play a critical role in several areas:
Population Health Management: Identifying high-risk individuals in a population for targeted intervention.
Clinical Decision Support: Assisting physicians in diagnosing and treating patients.
Resource Allocation: Prioritizing care and resources based on patient risk, including cost and utilization prediction.
Healthcare Policy: Informing policy and guideline development
. . .
Population Models
These models, typically utilized by payers and other organizations managing large populations, tend to target understanding disease burden and underlying diseases not yet realized, predicting risk (often in terms of future costs/spend), predicting or understanding utilization trends, and stratification of groups of people for case management, pop health management, etc. Many times, these are case-mix models focused on financial risk adjustment, effectively used in budget planning and capitation rate setting.
These are the 3 most common risk scores/groupers I’ve encountered in population health, though certainly there are many more. Their main strength lies in predicting financial risk from a payer's perspective, rather than clinical events:
The Johns Hopkins Adjusted Clinical Groups (ACG) System: This model, developed by the Johns Hopkins Bloomberg School of Public Health, uses patients' diagnoses and demographic data to predict healthcare utilization. The ACG System accounts for multiple simultaneous health conditions and the compounding complexity they bring, providing a more holistic view of patient health. This model considers the entirety of a patient's disease burden. It is beneficial for its comprehensive view, but its complexity can sometimes make it less transparent.
DXCG now by Cotiviti: DXCG models were originally developed in partnership with the U.S. Centers for Medicare & Medicaid Services (CMS) and served as the foundation for the HCC model that CMS still uses as the basis for risk-adjusted healthcare payments today. These models use diagnosis and pharmaceutical data for risk adjustment. They are known for their robustness, but their reliance on specific data may limit their use in data-poor settings. these models leverage both medical and pharmacy claims to identify high-risk patients. DXCG models are known for their comprehensiveness, covering the clinical spectrum from morbidity, comorbidity, and severity assessment. In 2022, DXCG celebrated its 25th anniversary.
Milliman Advanced Risk Adjusters (MARA): These proprietary models utilize demographic, clinical, and pharmaceutical data for risk adjustment. They're comprehensive but may be less transparent due to their proprietary nature. These models combine diagnoses and drug data to predict costs. What sets MARA apart is the use of Hierarchical Condition Categories (HCCs), allowing the model to capture the intricacies of patient condition hierarchies.
Let’s not forget all the proprietary models by other vendors. Many, many other vendors offer proprietary risk adjustment models, each with their unique methodologies and focus areas. For instance, 3M's Clinical Risk Groups (CRGs) lean towards chronic disease management, while Optum's Impact Pro model zeroes in on avoidable costs. HBI Solutions and other population health vendors have predictive models for all kinds of conditions and events using a combination of claims, enrollment, clinical, lab data, etc.
How do they stack up against each other?
Accuracy varies based on the use case and available data. DXCG models are robust in predicting costs in a general population, while the ACG system has been said to shine when considering multimorbidity. Milliman's MARA offers corresponding category groupers that I am a fan of.
The Society of Actuaries (SOA) validates these and others on occasion, but performance can vary based on the specific health outcome and population being analyzed. Here is a SOA report from 2016 evaluating claims-based risk scoring models.
Clinical Models
Some scores/calculators are more clinical decision support-focused (typically specific conditions/events). Note: I am not a clinician and therefore cannot speak to these, but they are examples of ones that I have either seen being used, heard about in my career, or have had to create analytics/calculations for:
CHADS2 and CHA2DS2-VASc Score: This model predicts the risk of stroke in patients with atrial fibrillation. It is simple and widely used, but it does not consider all potential risk factors.
Wells Criteria for DVT: Used for predicting the likelihood of deep vein thrombosis (DVT) and pulmonary embolism (PE). It is easy to use, but it requires clinical judgment, which can introduce bias.
Framingham Risk Score: Predicts the 10-year risk of heart disease. It is widely recognized, but it may not accurately predict risk in all populations. Note: there are several distinct Framingham risk models.
Apache II Score: Used for predicting mortality in ICU patients. It's comprehensive, but it may not fully account for individual patient variations.
HAS-BLED Score: Assesses the risk of bleeding in patients taking anticoagulants. It is said to be useful for treatment planning, but it doesn't consider all potential bleeding risk factors.
LACE Index for Readmissions: Predicts 30-day readmission or death in patients on medicine and surgery wards, with LACE being an Acronym for (L= Length patient Stay in the hospital, A= Acuity of Admission of the patient in the hospital, C= Comorbidity and E= Emergency Visit)
SOFA (and qSOFA) Score for Sepsis: Per ASPRHHS, “the Sequential Organ Failure Assessment (SOFA) score is a scoring system that assesses the performance of several organ systems in the body (neurologic, blood, liver, kidney, and blood pressure/hemodynamics) and assigns a score based on the data obtained in each category. The higher the SOFA score, the higher the likely mortality.” The qSOFA is the “quick” version.
These are a very, very, very few examples of many similar scoring/calculation tools out there for various conditions and/or events.
A resource to look up these clinical models to understand how they’re calculated and what factors they use (and calculate some scores interactively) is MDCalc. For example, pictured below is the LACE Index scoring tool.
I don’t recommend any non-clinicians use this for “prediction” in any sense, but it is helpful to see what kind of values they pull into different models, how they impact the ultimate score, and which values mean what.
. . .
Considerations When Using Any Model
Did I mention there are many, many (many) different models out there?
While these models are extremely valuable, they are not without limitations. Here are a few considerations:
Data Quality: The accuracy of predictions is highly dependent on the quality of the input data. How much of the population was included when developing the model? How applicable is that to your population? Like many things, the smaller the sample and the potential limitations of what kind of data is included could have an impact on the model. For instance, if a model was developed for Medicare patients that tend to be over 65+ and have multiple chronic conditions, it may not be as good to use on a Medicaid population. My article on Towards Data Science about bias in healthcare talks about this in great detail.
Model Fit: Not all models will work well for all populations or conditions. Model performance should be evaluated for the specific use case. Has the model been independently validated, peer-reviewed, etc? Has it been thoroughly tested? Does it adjust over time for drift in the data, process, or otherwise?
Ethical Considerations: Care must be taken to ensure that the use of these models does not inadvertently lead to healthcare inequities. For instance, attention has lately been on the lack of diversity and patient representation in large claim datasets, clinical trial data, etc. These are often the datasets models are developed from.
What data does it use? Is it using only claims data? Clinical data (with or without notes information)? Is it limited to only considering certain inputs? For instance, the LACE Index only considers 4 important factors that may have been found in the past to lead to readmissions, but certainly, there are others. Like I always advocate for with data and analytics, you must understand the rationale, context, and limitations of any model.
These are just a few important considerations. What else would you add?
. . .
Actionable Idea of the Week:
In choosing a model, the question is not about which is the best, but rather which is the best for your specific scenario. This involves understanding the nuances, evaluating how the current models you’re using work (or don’t) for your use case, and being more aware of what questions to start asking about the models and their development (for instance, the diversity and population coverage in the training data).
This week, challenge yourself with the following steps as it pertains to models you may be using but don’t fully know the details about and thereby take for granted or trust blindly:
Educate and Dive Deeper: Spend an hour or two delving into one or two models you're using now, or that you’re less familiar with. Research can illuminate nuances that might make a particular model more appropriate for a specific patient group or scenario. What data was it built upon? Who WON’T it work well for? How “accurate” is it?
Audit & Feedback: Take one case from your recent past where a predictive risk model was used and evaluate its accuracy. What worked, and what didn't, and why? Is there room for refining the process knowing this?
Advocate for Equity: See to understand what data models were built upon, and more importantly, what data was NOT present. Ensure that the models in use don't perpetuate healthcare inequities. If gaps are evident, raise awareness. Be the voice that prompts a more inclusive approach.
And if you’re in the business of creating models, these questions (and a whole lot more) apply to you too. If you fall into this bucket, I recommend my Guide on doing better analysis and creating more solid approaches using the 5W1H method.
. . .
See you next week!
-Stefany
2 more ways I can help you:
Like my content?
If you want to learn more about health data quickly so you can market yourself, your company, or just plain level up your health data game, I recommend subscribing to this newsletter and checking out my free Guides. Courses and more resources are coming soon, so check back often!
Want to work together?
I work with healthtech startups, investors, and other health organizations who want to transform healthcare and achieve more tangible, equitable outcomes by using data in new ways. Book some time with me to talk health data, advice for healthtech startup and investment, team training+workshops, event speaking, or Fractional Analytics Officer support + analytics advisory.
Follow me on LinkedIn, Twitter, and Medium to stay up-to-date on free resources and announcements!