An overview of the Case Mix Programme (CMP) Dataset (Version 3.1) is provided below:
An overview of the Case Mix Programme (CMP) process for the collection, entry, submission, validation, analysis and reporting of CMP data is shown in the diagram below:
Potential quality indicators are presented in a funnel plot format.
The grey lines represent control limits at two standard deviations (2SD) represented by a dashed line, and three standard deviations (3SD) represented by a solid line. Selected units are highlighted in orange.
As the number of admissions increases, the precision with which a result can be calculated increases.
If variation between results is random then on average 95% of points should lie within the inner control limits (2SD) and 99.8% should lie within the outer control limits (3SD).
Example – Hospital mortality
If the model is perfectly predicting hospital mortality, we expect the points to form a funnel shape centred on a mortality ratio of one (1.0).
Delayed Discharges - 12 and 24 hours
Please note the potential quality indicators relating to delayed discharges are not displayed as funnel plots. For these indicators, the dashed line indicates the 5% of critical care units with the highest rates.
CMP Quality Indicator Dashboard
The Case Mix Programme (CMP) Quality Indicator Dashboard summarises the results of each quality indicator for a single NHS adult, general critical care unit. Note that the dashboard can only be viewed if a specific unit is selected from the drop-down menus at the top of the page, for all other selections (hospital, trust, network or SHA) nothing will be shown.
Each row of the dashboard can be viewed as a 'slice' through the corresponding funnel plot (see below) showing the unit's own value for that potential quality indicator relative to the comparator value and two and three standard deviation lines.
For Hospital mortality, the comparator value is one (1.0), corresponding to observed acute hospital mortality equal to that expected from the ICNARC (2013) model.
For Unit-acquired infection in blood, Non-clinical transfers (out), Out-of-hours discharges to the ward, Daytime discharges to the ward , Delayed discharges (24 hour delay) and Unplanned readmissions within 48 hours, the agreed threshold value is based on the median across all participating CMP units in 2012/13 as published by the Clinical Reference Group (CRG) for specialised commissioning .
For Unit-acquired MRSA, Non-clinical transfers (out), Out-of-hours discharges to the ward (not delayed), Delayed discharges (12 hour delay) and Unplanned readmissions within 48 hours, the comparator value is the mid line (i.e. the overall percentage in all critical care units in the CMP for the time period of the report).
If the unit's value (orange line) falls within the light green or dark green regions of the dashboard, this corresponds to a value below the lower two standard deviation or three standard deviation lines of the funnel plot, respectively. If the unit's value falls within the dark orange or red regions, this corresponds to a value above the upper two standard deviation or three standard deviation lines, respectively.
Note that in some cases (particularly for small sample sizes or where an indicator exhibits a large amount of variation across critical care units), the green and/or orange/red regions may not be present.
The Clinical Reference Group (CRG) for adult critical care is responsible for specialised commissioning for critical care services.
ICNARC is working in conjunction with the CRG for Adult Critical Care to support specialised commissioning through the provision of data analyses from participating CMP units in a publically available format.
This information forms part of the Annual Quality Report in the form of a ‘dashboard’ for each critical care unit, indicating whether or not they meet the agreed threshold for each indicator.
Please note that some of the indicators provided in previous Annual Quality Reports have been amended/replaced by indicators prescribed by the CRG for adult critical care. Where relevant, we have also created new indicators, not currently prescribed by the CRG and retained our own CMP defined indicators.
For more information on the Clinical Reference Group for adult critical care, please follow the link below: https://www.england.nhs.uk/commissioning/spec-services/npc-crg/group-d/d16/ (external site)
ICNARC uses the ICNARC risk prediction model to calculate the risk of acute hospital mortality.
The ICNARC model was developed using data from over 200,000 admissions in the CMP Database (Harrison et al, 2007). Regular recalibration ensures that each critical care unit is being compared with current CMP data. To improve the fit of the model to the case mix of patients admitted to NHS adult, cardiothoracic critical care units, ICNARC recalibrated the ICNARC (Cardio 2013) model specifically to data from this unit type.
Please note: Data in this report may differ from those reported in participating critical care unit’s comparative reports (Data Analysis Reports) for the same period.
For more information on the ICNARC risk prediction model including inclusion and exclusion criteria, please follow the link below:
An outlier is a result that is statistically significantly further from the expected comparator value than would usually occur by chance alone.
ICNARC have developed guidance to ensure that potential outliers are identified through the processes of national clinical audit. The guidance document sets out the actions that ICNARC takes when data indicate that results for a site significantly deviate from the expected value.
ICNARC's guidance on the detection and management of outliers is based on Department of Health recommendations on the 'Detection and management of outliers' and supersedes ICNARC’s previous policy.
For more information on the detection and management of outliers, please follow the link below:
Detection and management of outliers - guidance (external link)
The ICNARC model was most recently recalibrated in July 2013 using data from 12,303 admissions to 5 NHS, adult cardiothoracic critical care units from January 2011 to December 2012.
For the purpose of the Case Mix Programme (CMP) Annual Quality Report 2013/14, the fit of the model was assessed, based on 8,995 admissions to 6 NHS, adult cardiothoracic critical care units using the following methods:
The c index (area under the ROC curve) was 0.891 (95% confidence interval 0.877 to 0.904. Calibration was generally very good (see below). Brier’s score was 0.051 and Shapiro’s R was 0.834. The ROC curve and calibration plot are shown below:
Calculation of funnel plots
Funnel plots were calculated using the methods of Spiegelhalter (2005).6 Over-dispersion was adjusted for by estimating a multiplicative over-dispersion factor for each potential quality indicator based on 10% Winsorisation.6,7 The estimated over-dispersion factors were: Hospital mortality 1.7; Hospital mortality - risk of death less than 20% 1.6; Hospital mortality - risk of death greater or equal to 20% 1.0; Unit-acquired MRSA 1.5; Unit-acquired infection in blood 2.2; Non-clinical transfers (out) 1.8; Out-of-hours discharges to the ward 1.7; Daytime discharges to the ward 1.7; Out-of-hours discharges to the ward (not delayed) 2.9; Unplanned readmissions within 48 hours 1.6.
1. Harrell FE, Califf RM, Pryor DB, et al. Evaluating the yield of medical tests. JAMA 1982; 247:2543-6.
2. Hanley JA, McNeil BJ. The meaning and use of the area under the receiver operating characteristics (ROC) curve. Radiology 1982; 143:29-36.
3. Kramer AA, Zimmerman JE. Assessing the calibration of mortality benchmarks in critical care: the Hosmer-Lemeshow test revisited. Crit Care Med 2007; 35:2052-6.
4. Brier GW. Verification of forecasts expressed in terms of probability. Monthly Weather Review 1950; 75:1-3.
5. Shapiro AR. The evaluation of clinical predictions. N Engl J Med 1977; 296:1509-14.
6. Spiegelhalter D. Funnel plots for comparing institutional performance. Stat Med 2005; 24:1185-202.
7. Spiegelhalter D. Handling over-dispersion of performance indicators. Qual Saf Health Care 2005; 14:347-51.