Instruct: Identify the main category of Medrxiv papers based on the titles and abstracts
Query: The Generalizability of Clinical Prediction Models for Patients with Acute Coronary Syndromes: Results from Independent External Validations PurposeIt is increasingly recognized that clinical prediction models (CPMs) often do not perform as expected when they are tested on new databases. Independent external validations of CPMs are recommended but often not performed. Here we conduct independent external validations of acute coronary syndrome (ACS) CPMs.

MethodsA systematic review identified CPMs predicting outcomes for patients with ACS. Independent external validations were performed by evaluating model performance using individual patient data from 5 large clinical trials. CPM performance with and without various recalibration techniques was evaluated with a focus on CPM discrimination (c-statistic, % relative change in c-statistic) as well as calibration (Harrells Eavg, E90, Net Benefit).

ResultsOf 269 ACS CPMs screened, 23 (8.5%) were compatible with at least one of the trials and 28 clinically appropriate external validations were performed. The median c statistic of the CPMs in the derivation cohorts was 0.76 (IQR, 0.74 to 0.78). The median c-statistic in these external validations was 0.70 (IQR, 0.66 to 0.71) reflecting a 24% decrement in discrimination. However, this decrement in discrimination was due mostly to narrower case-mix in the validation cohorts compared to derivation cohorts, as reflected in the median model based c-statistic [0.71 (IQR 0.66 to 0.75). The median calibration slope in external validations was 0.84 (IQR, 0.72 to 0.98) and the median Eavg (standardized by the outcome rate) was 0.4 (IQR, 0.3 to 0.8). Net benefit indicates that most CPMs had a high risk of causing net harm when not recalibrated, particularly for decision thresholds not near the overall outcome rate.

ConclusionIndependent external validations of published ACS CPMs demonstrate that models tested in our sample had relatively well-preserved discrimination but poor calibration when externally validated. Applying  off-the-shelf CPMs often risks net harm unless models are recalibrated to the populations on which they are used.
26
The categories of this paper are: systems
63
The categories of this paper are: pain
57
The categories of this paper are: primary
30
The categories of this paper are: radiology
51
The categories of this paper are: education
5
The categories of this paper are: health
46
The categories of this paper are: therapeutics
66
The categories of this paper are: dentistry
65
The categories of this paper are: oral
12
The categories of this paper are: genomic
20
The categories of this paper are: psychiatry
25
The categories of this paper are: improvement
49
The categories of this paper are: intensive
64
The categories of this paper are: ophthalmology
40
The categories of this paper are: geriatric
38
The categories of this paper are: hiv