Clinical prediction models for risk-adjusted outcomes in South African surgical patients

Abstract

Background National clinical data on perioperative care in South Africa are scarce. There is both an urgent need, and the imminent opportunity, to increase the body of evidence necessary to inform on initiatives to improve safety, affordability and access to surgical- and anaesthesia care in this country. Clinical prediction models are a useful way to present factors that predict a specific endpoint, and the relationships of these factors in influencing the endpoint. Such summarised information is important for perioperative clinicians and teams to understand how their circumstances and their practice influence a patient’s outcome after surgery, and how this influence, and the outcome, compare to teams in different circumstances or institutions. Developing clinical prediction models is an exercise in defining and identifying predictors and endpoints that should form part of a core set of measures for research on perioperative care. It is crucial to validate clinical prediction models in settings other than where it was developed before it is implemented. Prediction models may require updating before it can be generalisable. The aim of this thesis is to report on clinical prediction model development in two surgically heterogeneous South African cohorts: i) a public sector cohort; the South African dataset from the African Surgical Outcomes Study (ASOS); and ii) a private sector cohort, from data gathered for the purpose of model development, in patients presenting for elective non-cardiac surgery in a single private hospital. Methods Data from two cohorts of patients that differ with regards to the sample population, and the healthcare sector, were used to develop two separate clinical prediction models. A clinical prediction model with in-hospital mortality as endpoint was developed in the public sector cohort. A prediction model with healthcare resource use as endpoint was developed from a self-assessment questionnaire in the private sector cohort. Using clinical judgement, predictors for the prediction models were identified from univariate regression analysis and subsequent forward stepwise regression techniques. The prediction models were assessed for performance regarding calibration, discrimination and clinical usefulness, and were internally validated by fitting to a bootstrap sample. The prediction model that was developed from the ASOS South Africa cohort was validated in the cohort of patients participating in the South African Surgical Outcomes Study, which is a temporally separate dataset containing data collected in 2014. The possibility of validating the Surgical Outcomes Risk Tool, an established prediction tool, in the ASOS South Africa cohort, was investigated. There is currently no data available for external validation of the prediction model developed in the private sector cohort. Results During prediction model development, important variables (predictors and endpoints) were identified that should form part of a core dataset. The ASOS South Africa prediction model was developed with postoperative in-hospital mortality, censored at thirty days, as the endpoint. The predictors included in the prediction model were largely related to the risk inherent to the urgency, severity and type of surgical procedure. The private sector prediction model was developed with the cost of hospital admission, excluding fees, as endpoint. The predictors included were the type of surgery and predictors defined from patient-reported information. Although both prediction models performed fairly well with regard to calibration, discrimination and clinical usefulness, the prediction models will require validation in cohorts of patients representing a different South African population. It is expected that the prediction models will require adjustment or updating after external validation. The definitions of predictors will also have to be reconsidered when validating these prediction models in cohorts from other settings. Conclusion During the development of the clinical prediction models, predictor definitions were investigated. Variables (predictors and endpoints) should be defined in such a way as to align with international classification systems, since these are used to ‘code’ variables in electronic health information systems to enable aggregation of data. The advantages of external validation of clinical prediction models, and the subsequent prediction model updating, would be: the opportunity to further refine the definitions of candidate predictors to enable international comparisons; the potential to include health economic measures to inform on the cost-effectiveness of surgery; and the chance to define and include patient-reported measures in the core data set. The result may well be that evidence gathered in this way would assist in developing strategies for optimal delivery of perioperative care to the entire South African population. Doctors, and their patients, will have to voluntarily participate in national multicentre research projects to gather evidence on perioperative care in South Africa. One has to consider the additional burden the collection of perioperative data would entail, and how such ‘citizen scientists’ would be motivated to participate.