🧵 In Summary

Prediction models are not “set and forget.” They must be constantly checked, updated, and re-evaluated—before and after clinical use.

With thoughtful updating, we can keep models useful and safe over time.

📄 Read our commentary: kwnsfk27.r.eu-west-1.awstrack.me/L0/https:%2F...

🚨 What About After Implementation?

Once a model is used in practice, it changes patient outcomes (e.g., high-risk patients get treated and avoid the outcome).

This shifts the relationship between risk factors and outcomes. Updating without accounting for this is risky.

⚖️ Fairness Under Threat

Temporal drift can impact some groups more than others.

Even worse, updating models doesn’t guarantee fairness. It can reduce—or worsen—performance in minority groups.

We need to monitor algorithmic fairness, not just accuracy.

💻 The Catch? Complexity

Dynamic updates need:
Continuous data
More computing power
Digital infrastructure in place

Also, no method kept the original performance in later years. So, updating helps—but doesn’t fix everything.

In our commentary, we have expanded on the findings of the study, the impacts of temporal drift and the importance of addressing it.

A recent study looked at which methods could be effective at addressing this drift doi.org/10.1016/j.bp...

They found that without updating, model performance deteriorated and dynamic updating might be able to address this.

🎯 Why It Matters

One form of drift—calibration drift—means predicted risks no longer match actual outcomes.

📉 High-risk patients might be missed.
📈 Low-risk patients might be overtreated.

This can harm patients and erode trust in predictive tools.

📣 Bottom line:
People missed by CHR-P services experience a similar prodrome to those who are detected. We need better ways to detect them.

📄 Read the full preprint: doi.org/10.1101/2025...

#Psychosis #MentalHealth #EHR #NLP #EarlyIntervention

📊 Key Findings:
Presence of psychosis prodrome: 85% in both groups
Duration of prodrome: ~18 months in both groups
Symptoms at first presentation: no differences
Symptoms across the prodrome: no differences

Using electronic health records and natural language processing (NLP), we extracted 65 prodromal features (symptoms & substance use) from 1,545 FEP patients.

We compared the presence, duration, first presentation and frequency of prodromal symptoms.

Could it be that those missed (DET-) don’t show early warning signs?

We tested this by comparing DET- patients to those detected (DET+) by CHR-P services before psychosis onset.

Do they share the same prodrome?

Early detection of psychosis through CHR-P (clinical high risk for psychosis) services can allow for preventive care — but most patients are not detected before their first episode (FEP). Why?

Today postdoctoral researcher Dr Dominic Oliver, who works on the Data Science Theme at our BRC, highlights how new predictive tools can help prevent early signs and symptoms of psychosis escalating.

Read a review of the programme of research: tinyurl.com/y4nj6ha6
@domapoliver.bsky.social