bmj...13 care, university medical centre utrecht, str. 6.131, po box 85500, utrecht, the netherlands...

Confidential: For Review O

nly

External validation of prognostic models to predict the risk

of developing gestational diabetes

Journal: BMJ

Manuscript ID BMJ.2016.033135

Article Type: Research

BMJ Journal: BMJ

Date Submitted by the Author: 23-Apr-2016

Complete List of Authors: Lamain-de Ruiter, Marije; Universitair Medisch Centrum Utrecht, Woman and Baby Kwee, Anneke; Wilhelmina Children's Hospital, University Medical Centre, Utrecht, Department of Obstetrics Naaktgeboren, Christiana; University Medical Center Utrecht, Julius Center

for Health Sciences and Primary Care Franx, Arie; University Medical Center Utrecht, Obstetrics and gynecology Moons, Karel; Julius Center for Health Sciences and Primary Care, Epidemiology Koster, Maria; University Medical Center Utrecht, Department of Obstetrics; Erasmus MC de Groot, Inge; Tweesteden Ziekenhuis Vestiging Sint Elisabeth, Livive, Centre for Obstetrics Evers, Inge; Meander Medisch Centrum, Obstetrics Groenendaal, floris; University Medical Centre Utrecht, Department of Neonatology Hering, Yolanda; Zuwe Hofpoort Ziekenhuis, Obstetrics

Huisjes, Anjoke; Gelre Hospitals, Perinatology Kirpestein, Cornel; Ziekenhuis Rivierenland, Obstetrics Monincx, Wilma; Sint Antonius Ziekenhuis, Obstetrics Siljee, Jacqueline; Rijksinstituut voor Volksgezondheid en Milieu, Centre for Infectious Diseases Research, Diagnostics and screening (IDS) Van 't Zelfde, Annewil; Midwifery practice \'Verloskundigen Amersfoort Van Oirschot, Charlotte; Tweesteden Ziekenhuis Vestiging Sint Elisabeth, Obstetrics Vankan-Buitelaar, Simone; Midwifery practice 'GCM' Vonk, Mariska; Midwifery practice 'Het Wonder' Wiegers, Trees; Nederlands Instituut voor Onderzoek van de

Gezondheidszorg Zwart, Joost; KNOV, guidelines

Keywords: first trimester, gestational diabetes, external validation, prognostic model, head to head comparison

https://mc.manuscriptcentral.com/bmj

BMJ


nly

Page 1 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

23 April 2016 1

TITLE PAGE 1

External validation of prognostic models to predict the risk of developing gestational 2

diabetes 3

Lamain-de Ruiter M1, Kwee A

2, Naaktgeboren CA

3, Franx A

4, Moons KGM

5, Koster MPH

6, on 4

behalf of the RESPECT study group 5

23 April 2016 6

Affiliations 7

1 PhD student, Department of Obstetrics, Division Woman and Baby, University Medical 8

Centre Utrecht, KE.04.123.1, PO BOX 85090, 3508 AB, Utrecht, The Netherlands 9

2 Obstetrician, Department of Obstetrics, Division Woman and Baby, University Medical 10

Centre Utrecht, KE.04.123.1, PO BOX 85090, 3508 AB, Utrecht, The Netherlands 11

3 Assistant professor of Clinical Epidemiology, Julius Centre for Health Sciences and Primary 12

Care, University Medical Centre Utrecht, Str. 6.131, PO BOX 85500, Utrecht, The Netherlands 13

4 Professor of Obstetrics, Department of Obstetrics, Division Woman and Baby, University 14

Medical Centre Utrecht, Utrecht, KE.04.123.1, PO BOX 85090, The Netherlands 15

5 Professor of Clinical Epidemiology, Julius Centre for Health Sciences and Primary Care, 16

University Medical Centre Utrecht, Str. 6.131, PO BOX 85500, Utrecht, The Netherlands 17

6 Assistant professor of Obstetrics & Gynaecology, Department of Obstetrics, Division 18

Woman and Baby, University Medical Centre Utrecht, KE.04.123.1, PO BOX 85090, 3508 AB, 19

Utrecht, The Netherlands & Department of Obstetrics and Gynaecology, Erasmus Medical 20

Centre, PO BOX 2040, 3000CA, Rotterdam, The Netherlands 21

22

23

Page 2 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

23 April 2016 2

Collaborators in the RESPECT study group 24

de Groot I7, Evers IM

8, Groenendaal F

9, Hering Y

10, Huisjes AJM

11, Kirpestein C

12, Monincx 25

WM13

, Siljee JE14

, Van ’t Zelfde A15

, Van Oirschot CM16

, Vankan-Buitelaar S17

, Vonk MAAW18

, 26

Wiegers TA19

, Zwart JJ20

27

28

Affiliations 29

7 Midwife, ‘Livive’, Centre for Obstetrics, Tilburg, The Netherlands

30

8 Obstetrician, Department of Obstetrics, Meander Medical Centre, Amersfoort, The 31

Netherlands 32

9 Neonatologist, Department of Neonatology, Division Woman and Baby, University Medical 33

Centre Utrecht, Utrecht, The Netherlands 34

10 Midwife, Department of Obstetrics, Zuwe Hofpoort Hospital, Woerden, The Netherlands 35

11 Obstetrician, Department of Obstetrics, Gelre Hospital, Apeldoorn, The Netherlands 36

12 Midwife, Department of Obstetrics, Hospital Rivierenland, Tiel, The Netherlands 37

13 Obstetrician, Department of Obstetrics, St. Antonius Hospital, Nieuwegein, The 38

Netherlands

39

14 Senior researcher, Centre for Infectious Diseases Research, Diagnostics and Screening 40

(IDS), National Institute for Public Health and the Environment (RIVM), Bilthoven, The 41

Netherlands 42

15 Midwife, Midwifery practice ‘Verloskundigen Amersfoort’, Amersfoort, The Netherlands 43

16 Obstetrician, Department of Obstetrics, St Elisabeth Hospital, Tilburg, The Netherlands 44

17 Midwife, Midwifery practice ‘GCM’, Maarssenbroek, The Netherlands 45

18 Midwife, Midwifery practice ‘Het Wonder’, Houten, The Netherlands 46

Page 3 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

23 April 2016 3

19 Senior researcher, Netherlands Institute for health services research (NIVEL), Utrecht, The 47

Netherlands 48

20 Obstetrician, Department of Obstetrics, Deventer Hospital, Deventer, The Netherlands 49

50

Address for correspondence 51

Name: Marije Lamain-de Ruiter 52

Phone number: +31-6-55 23 46 52 (work) 53

E-mail address: [email protected] 54

55

Grants 56

This study has been conducted with the support of The Netherlands Organization for Health 57

Research and Development (project nr 50-50200-98-060). 58

59

Word count 60

Abstract: 267 61

Main text: 3257 62

Page 4 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

23 April 2016 4

Abstract 63

Objectives 64

To perform an external validation and direct comparison of published prognostic models for 65

early prediction of the risk of developing gestational diabetes (GDM), including predictors 66

applicable in the first trimester of pregnancy. 67

Design 68

External validation of all published prognostic models in a large scale prospective cohort 69

study. 70

Setting 71

31 independent midwifery practices and 6 hospitals in the Netherlands. 72

Participants 73

Women were included in the first trimester of pregnancy, <14 weeks of gestational age. 74

Women with pre-existing diabetes mellitus of any type were excluded. 3,723 women were 75

included for analysis of which 181 (4.9%) developed GDM in pregnancy. 76

Main outcome measures 77

Discrimination of the prognostic models was assessed by the C statistic, and calibration by 78

calibration plots. Findings were reported conform the TRIPOD statement for validation of 79

prognostic models. 80

Results 81

A systematic literature review identified 14 published prognostic models for GDM of which 82

12 models could be validated in our cohort. The C statistic ranged from 0.67 to 0.78. 83

Calibration plots showed that most models were well calibrated. The four models with the 84

highest C statistic (all >0.75) often included maternal age, maternal body mass index, history 85

Page 5 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

23 April 2016 5

of GDM, ethnicity, and family history of diabetes as predictors. Prognostic models had a 86

similar performance in a subgroup of nulliparous women. 87

Conclusions 88

In this external validation study, most of the published prognostic models for GDM show 89

acceptable discrimination and calibration. The four models with the highest discriminative 90

abilities in our population, and which also perform well in a subgroup of nulliparous women, 91

are easy models to apply in clinical practice and therefore deserve further evaluation 92

regarding its clinical impact. 93

94

What is already known on this subject 95

- Gestational diabetes is an increasingly common complication of pregnancy and pregnancy outcome 96

can be improved through early screening, diagnosis and treatment. 97

- Numerous prognostic models for estimating the risk of developing gestational diabetes have been 98

developed, but an external validation and direct comparison in an independent large cohort of all 99

published models is lacking. 100

What this study adds 101

- This external validation study shows that in a direct comparison most published first trimester 102

prognostic models for GDM have an acceptable discrimination and good calibration. 103

- Well-performing first trimester prognostic models for GDM may be considered for implementation 104

in routine clinical care. 105

- Findings are reported conform the TRIPOD statement for validation of prognostic models. 106

107

Page 6 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

23 April 2016 6

Introduction 108

In the field of obstetrics, the number of publications on prognostic models has more than 109

tripled in the last decade1, which reflects an increasing interest in risk-based medicine. Risk-110

based medicine aims to provide the most appropriate care to each patient, often guided by 111

outcome risk estimates based on individual patient characteristics, test results or even 112

genetic information.2 113

As a result of the obesity pandemic, the incidence of gestational diabetes (GDM), notably 114

occurring in the second or third trimester, is rising and is increasingly contributing to 115

perinatal complications, such as macrosomia, shoulder dystocia, caesarean section, and 116

neonatal hypoglycaemia.3,4

Moreover, long term sequelae of GDM are type II diabetes in the 117

mothers and obesity in their offspring.5,6

Diagnosis and treatment of GDM have been proven 118

to improve pregnancy outcomes.7,8

Some guidelines propose a population strategy for 119

diagnosing GDM (i.e. an oral glucose tolerance test (OGTT)) in each pregnant woman, where 120

others opt for a high risk strategy, an approach in which testing for GDM is only performed 121

in women with known risk factors. Both strategies include oGTTs in substantial numbers of 122

women, most of which will lead to negative results, and therefore pose a too high burden to 123

patients as well as health care resources.9 Accurate prognostic models for the risk of 124

developing GDM early in pregnancy may allow one to discriminate the high-risk from the 125

low-risk pregnancies, and to move towards more tailored care in pregnancy. In particular, 126

this may result in fewer women undergoing a burdensome diagnostic test. 127

Several prognostic models for gestational diabetes have been developed. However, these 128

prognostic models are not commonly used in routine clinical care nor are they 129

recommended by current guidelines. This may be due to the fact that external validation of 130

Page 7 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

23 April 2016 7

these prognostic models are scarce,10–13

let alone that all these models have been directly 131

evaluated and compared based on their predictive accuracy in one single and independent 132

cohort by independent investigators. To acquire a fair comparison of their predictive 133

accuracy, and thus of their clinical value, it is essential to perform a head-to-head 134

comparison of all published prognostic models in one independent cohort.14–16

Thus, the aim 135

of our study was to perform the very first external validation and direct head-to-head 136

comparison of all published first trimester prognostic models for GDM, in a single 137

independent cohort. 138

Page 8 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

23 April 2016 8

Methods 139

Prognostic models identified by systematic literature review 140

We performed a systematic review in which we identified 14 published prognostic models 141

for GDM that are applicable in the first trimester of pregnancy and that only consist of 142

routine and easy to obtain measures [Lamain-de Ruiter M, Kwee A, Naaktgeboren CA, Franx 143

A, Moons KGM, Koster MPH. Prediction models for the risk of gestational diabetes: a 144

systematic review]. For proper external validation it is favourable that the exact definitions 145

of the predictors included in the developed prognostic model are known as well as how they 146

were measured. Although five of the author groups of the publications of these prognostic 147

models were contacted by email for additional information on intercepts, coefficients, and 148

definitions of predictors in the model, none of them responded. Despite this lack of 149

information, it was still possible to include four of these models in our head-to-head 150

validation study, but one model had to be excluded due to missing intercept and 151

coefficients.17

Moreover, one prognostic model was excluded from analysis for utilizing 152

maternal abdominal circumference and diagnosis of polycystic ovary syndrome (PCOS) as 153

predictors, which we did not collect in our validation cohort and for which was also no proxy 154

variable available.18

Thus, a total of 12 prognostic models remained for external validation in 155

the current study.19–28

In Appendix A the equations of these 12 prognostic models as applied 156

in our cohort are shown. 157

None of the authors were involved in the development of any of these models. The results 158

are reported conform the tripod statement for transparent reporting of validation of 159

prognostic models (appendix D).29,30

160

161

Page 9 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

23 April 2016 9

Study population of external validation cohort 162

We performed a large prospective multicentre cohort study (Risk EStimation for PrEgnancy 163

Complications to provide Tailored care (RESPECT) study) to validate the 12 prognostic 164

models for GDM. From December 2012 through January 2014 we included pregnant women 165

at their initial prenatal visit (i.e. <14 weeks of gestational age (GA)) in 31 independent 166

midwifery practices (primary care) and 6 hospitals (secondary/tertiary care) in the central 167

region of the Netherlands. We excluded women suffering from any type of diabetes mellitus 168

(DM) from the cohort. During the course of their pregnancy, participants received routine 169

antenatal care according to Dutch clinical guidelines. 170

This study was approved by the medical ethics committee of the University Medical Centre 171

Utrecht (protocol number 12-432/C) and written informed consent was obtained from all 172

participants. 173

174

Predictor assessment 175

In table 1 we have summarised predictors that were included in the 12 selected prognostic 176

models. Predictors for GDM were all measured in the first trimester at initial prenatal visit by 177

caregivers or via a self-administered questionnaire. Detailed information on predictor 178

definition and measurement can be found in Appendix B. 179

180

Outcome assessment 181

GDM was diagnosed by a 75 grams two-hour oral glucose tolerance test (OGTT) between 24 182

and 28 weeks of gestation according to the WHO 1999 guidelines as the presence of either a 183

fasting glucose level of ≥7.0 mmol/L (126 mg/dl) or a glucose level of ≥7.8 mmol/L (140 184

Page 10 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

23 April 2016 10

mg/dl) after two hours.31,32

Women were offered an OGTT if risk factors or any signs of GDM 185

were present. Without risk factors or signs women were not tested and considered as not 186

having GDM. 187

Body mass index (BMI) in first trimester >30kg/m2, history of GDM, history of macrosomia 188

(birth weight above 95th

Dutch population centile)33

, family history of DM (first degree), non-189

western ethnicity, history of unexplained intrauterine foetal death, and polycystic ovary 190

syndrome (PCOS) were considered as risk factors for GDM. Polyhydramnios and macrosomia 191

were considered as possible signs of GDM. 192

For studies validating prognostic models there is no solid sample size recommendation, but a 193

minimum of 100 patients with events and at least 100 patients without events has been 194

suggested.34

195

196

Statistical Analysis 197

Predictor and outcome information was missing for some patients in the validation cohort 198

and these data were not missing completely at random, as can be derived from table 2. To 199

avoid biased validation of the models, we imputed the missing values using multiple 200

imputation.35

201

To start, we applied the “original” prognostic models, exactly as they were published, to our 202

study cohort when the full prediction rule, including its intercept, was available (appendix A). 203

Next, to allow for fair comparison of the prognostic models, we adjusted the intercepts of 204

the models to the cohort at hand, so that the mean predicted probability in each model was 205

equal to the observed outcome frequency. This is known as “recalibration-in-the-large” and 206

was performed by fitting a logistic regression model using the linear predictor as the only 207

Page 11 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

23 April 2016 11

covariate in a new model (and offsetting it so that only the intercept would be 208

estimated).36,37

We then performed “logistic recalibration” by fitting logistic regression 209

models using the linear predictor as the only covariate. This resulted in an updated 210

calibration slope and intercept.36,38

211

We validated the original and recalibrated models by calculating the predicted probabilities 212

of GDM for each individual and comparing these with their observed outcomes. We assessed 213

discrimination using Harrell’s C statistic, which is equivalent to the area under the ROC 214

curve.39

It verifies whether participants with a higher predicted risk for GDM are indeed 215

more likely to have the disease. 216

Calibration refers to the agreement of predicted probabilities with observed proportions and 217

was assessed using calibration plots. When a model has perfect calibration, the predicted 218

probabilities equal the observed proportions. Thus, when a model is well calibrated, the 219

calibration plot has an intercept of 0 and a slope of 1. Some calibration plots have fewer 220

than 10 points because it was not possible to split the predicted probabilities into 10 groups. 221

This was the case for models with only a few categorical variables were included (e.g. sum 222

score models) in which a limited probabilities were possible. 223

A history of GDM is an important predictor in most models, but obviously not applicable for 224

nulliparous women. Therefore, discrimination and calibration of all 12 models were also 225

assessed in a subgroup analysis of nulliparous women. 226

All analyses were carried out on each of the multiple imputed datasets and Rubin’s rules 227

were used to combine the results into summary estimates. Analyses were performed using 228

the mice and rms packages of R-3.1 for windows (http://cran.r-project.org). 229

Page 12 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

23 April 2016 12

Results 230

RESPECT cohort 231

3,723 women were included for analysis of which 1,655 (44%) were nulliparous (figure 1). 232

Table 2 shows the baseline characteristics of these women. GDM was diagnosed in 181 233

(4.9%) women, 33 (18%) of which needed insulin for glycaemic control. In the nulliparous 234

subgroup, 71 women (4.3%) developed GDM. 235

236

Calibration of the prognostic models 237

Three original publications provided the full prediction rule (Gabbay-Benziv 2014, Savona-238

Ventura 2013, and Van Leeuwen 2010) of which two models showed good calibration 239

(Gabbay-Benziv 2014, Van Leeuwen 2010) (figure 2). The model of Savona-Ventura et al. had 240

a poor calibration and tended to overestimate the risk of GDM. 241

Calibration plots were also drawn for each recalibrated model (figure 3). Most recalibrated 242

models showed good calibration, with the calibration line closely following the ideal 243

calibration line, except for four models. The models of Eleftheriades 2014, Naylor 1997, and 244

Tran 2013 seemed to overestimate the risk for women with an observed high risk of GDM, 245

whereas the model of Pintaudi 2014 seemed to underestimate the risk for these women. 246

When we compared the calibration plots of the original models with the recalibrated models 247

all three calibration plots improved. 248

249

Discrimination 250

C statistics for the original and recalibrated models ranged from 0.67 to 0.78 (table 3). The 251

four models with the highest C statistic (Gabbay-Benziv 2014, Nanda 2011, Teede 2011, and 252

Page 13 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

23 April 2016 13

Van Leeuwen 2010) included maternal age, BMI, history of GDM, ethnicity, and family 253

history of DM as predictors. The poorest discriminating models were the models containing 254

the fewest predictors (Eleftheriades 2014, Savona-Ventura 2013, and Tran 2013). 255

256

Subgroup analysis 257

Discrimination for nulliparous women was worse as compared to the overall population for 258

four prognostic models (Gabbay-Benziv 2014, Nanda 2011, Naylor 1997, and Teede 2011) 259

(table 3). For all other models the C statistic was higher for nulliparous compared to all 260

women. Calibration of the prognostic models was also acceptable to good in the nulliparous 261

subgroup (figure 4). 262

Page 14 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

23 April 2016 14

Discussion 263

Statement of principal findings 264

A total of 12 first trimester prognostic models for GDM were selected by comprehensive 265

review of literature and were compared head-to-head for their predictive accuracy in our 266

population-based cohort of 3,723 women. The model with the highest discriminative ability, 267

by Nanda et al., is any easy model to apply in clinical practice. Predictors in this particular 268

prognostic model (i.e. maternal age, BMI, ethnicity, history of GDM and history of 269

macrosomia) are easy to measure and their categorization of ethnicity is widely applicable. 270

Calibration was good for all models and improved by recalibrating the models to our 271

population. Although obstetric history is an important predictor in most models, the 272

prognostic models for GDM also performed well in nulliparous women. 273

274

Strengths and limitations 275

This has been the first external validation study that comprises almost all published first 276

trimester prognostic models for GDM in one single cohort study, allowing for head-to-head 277

comparison of these models. Our study had a large sample size and many cases of GDM. It 278

was performed using a prospective multicentre approach, and included an unselected 279

population of women from primary care (low-risk) as well as secondary/tertiary care (high-280

risk) within a geographically defined area. Additionally, missing data was handled by multiple 281

imputation, which is the most preferable method.34

282

However, some limitations of our study need to be addressed. First, according to Dutch 283

guidelines a high risk strategy was adhered. To prevent unnecessary testing in study 284

participants, women without predefined risk factors only underwent an OGTT in case of any 285

Page 15 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

23 April 2016 15

symptoms of GDM. This strategy may have led to an underestimation of GDM cases in low-286

risk women. A study on the performance of similar strategies estimated that 7.3% of GDM 287

diagnoses may have been missed.40

However, this possible underestimation is unlikely to 288

have influenced the discriminative ability of the validated models since the C statistic is a 289

rank order insensitive to systematic errors in calibration such as differences in outcome 290

incidence.41

Moreover, this potential underestimation is also unlikely to have influenced our 291

inferences on the predictive accuracy of the models since we have recalibrated the 292

models, which accounts for any differences in overall incidence between the original model 293

development studies and our external validation study. 294

A second limitation of our study was that we were not able to include two published 295

prognostic models in our external validation. For one model information on the prediction 296

rule was not available despite contacting the authors. The other study was published after 297

the start of data collection for our validation cohort and information on some predictors 298

(maternal abdominal circumference and presence of PCOS) was not collected. 299

300

Comparison with other studies 301

Validation studies on prognostic models for GDM are scarce and our study differs from the 302

few previously published validation studies10–13

since we have performed a head-to-head 303

comparison where others validated a single prognostic model or only a small selection of the 304

prognostic models for GDM. However, our findings are similar to the findings of these 305

external validation studies, except for the external validation of the Van Leeuwen 2010 306

model by Lovati et al.10

In that study, the Van Leeuwen 2010 model yielded a poor C statistic 307

(0.60), in contrast to the other external validation studies, including our current study, that 308

Page 16 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

23 April 2016 16

showed C statistics between 0.75 and 0.77.12,26

This might be due to the case-control study 309

design chosen by Lovati et al. in which it is not possible to adjust for observed outcome 310

frequency. 311

312

Clinical implications and conclusions 313

The use of accurate first trimester prognostic models for GDM will avoid the need to 314

perform an OGTT in all or many women, as is now recommended by various international 315

guidelines.42,43

A comparison of performance between the best discriminating prognostic 316

models and current strategies allows weighing the pros and cons (e.g. missed cases) and will 317

help to choose the model to be implemented into clinical practice. The decision which of the 318

four best models to implement in clinical practice may also depend on population 319

characteristics, the availability of predictors, and the incidence of GDM, and may therefore 320

be country- or region-specific. 321

Implementation of prognostic models for GDM early in pregnancy provides room for 322

preventive measures, i.e. lifestyle modification interventions such as diet and exercise 323

counseling.44,45

Additionally, metformin is likely to play a role in the prevention of GDM in 324

the near future.46,47

Early prevention, screening, diagnosis, and treatment of GDM when 325

necessary, can and will most likely reduce the rates of caesarean section, neonatal 326

hypoglycaemia and macrosomia and long term neonatal complications.4 327

To conclude, most of the 12 previously published prognostic models for GDM that have been 328

validated in this study show acceptable to good discrimination and calibration. Four models 329

outstand with C statistics of at least 0.75 (Gabbay-Benziv 2014, Nanda 2011, Teede 2011, 330

and Van Leeuwen 2010). We recommend that these four models will be further investigated 331

Page 17 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

23 April 2016 17

for implementation in clinical practice. The model with the highest discriminative ability by 332

Nanda et al. is an easy model to apply in clinical practice, as the model consists of 333

straightforward predictors: maternal age, BMI, history of GDM, history of macrosomia and 334

ethnicity. Once prognostic models for GDM are applied in routine clinical care, further 335

research is recommended on the effects on clinical impact, actual development of GDM and 336

subsequent pregnancy outcomes. 337

Page 18 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

23 April 2016 18

References 338

1. Kleinrouweler CE, Cheong-See FM, Collins GS, et al. Prognostic models in obstetrics: 339

available, but far from applicable. Am J Obstet Gynecol 2016; 214: 79–90.e36. 340

2. James JE. Personalised medicine, disease prevention, and the inverse care law: more 341

harm than benefit? Eur J Epidemiol 2014; 29: 383–90. 342

3. DeSisto CL, Kim SY, Sharma AJ. Prevalence estimates of gestational diabetes mellitus 343

in the United States, Pregnancy Risk Assessment Monitoring System (PRAMS), 2007-344

2010. Prev Chronic Dis 2014; 11: E104. 345

4. Metzger BE, Lowe LP, Dyer AR, et al. Hyperglycemia and adverse pregnancy outcomes. 346

N Engl J Med 2008; 358: 1991–2002. 347

5. Bellamy L, Casas J-P, Hingorani AD, Williams D. Type 2 diabetes mellitus after 348

gestational diabetes: a systematic review and meta-analysis. Lancet 2009; 373: 1773–349

9. 350

6. Clausen TD, Mathiesen ER, Hansen T, et al. High prevalence of type 2 diabetes and 351

pre-diabetes in adult offspring of women with gestational diabetes mellitus or type 1 352

diabetes: the role of intrauterine hyperglycemia. Diabetes Care 2008; 31: 340–6. 353

7. Alwan N, Tuffnell DJ, West J. Treatments for gestational diabetes. Cochrane database 354

Syst Rev 2009; 8: CD003395. 355

8. Viana LV, Gross JL, Azevedo MJ. Dietary intervention in patients with gestational 356

diabetes mellitus: a systematic review and meta-analysis of randomized clinical trials 357

on maternal and newborn outcomes. Diabetes Care 2014; 37: 3345–3355. 358

9. Werner EF, Pettker CM, Zuckerwise L, et al. Screening for gestational diabetes 359

mellitus: are the criteria proposed by the international association of the Diabetes and 360

Pregnancy Study Groups cost-effective? Diabetes Care 2012; 35: 529–35. 361

10. Lovati E, Beneventi F, Simonetta M, et al. Gestational diabetes mellitus: Including 362

serum pregnancy-associated plasma protein-A testing in the clinical management of 363

primiparous women? A case-control study. Diabetes Res Clin Pract 2013; 100: 340–364

347. 365

11. Van Leeuwen M, Opmeer BC, Zweers EJK, et al. External validation of a clinical scoring 366

system for the risk of gestational diabetes mellitus. Diabetes Res Clin Pract 2009; 85: 367

96–101. 368

12. Theriault S, Forest J-C, Masse J, Giguere Y. Validation of early risk-prediction models 369

for gestational diabetes based on clinical characteristics. Diabetes Res Clin Pract 2014; 370

103: 419–25. 371

13. Syngelaki A, Pastides A, Kotecha R, et al. First-Trimester Screening for Gestational 372

Diabetes Mellitus Based on Maternal Characteristics and History. Fetal Diagn Ther 373

2015; 38: 14–21. 374

14. Steyerberg EW, Moons KG, van der Windt DA, et al. Prognosis Research Strategy 375

(PROGRESS) 3: prognostic model research. PLoSMed 2013; 10: e1001381. 376

15. Justice AC. Assessing the Generalizability of Prognostic Information. Ann Intern Med 377

Page 19 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

23 April 2016 19

1999; 130: 515. 378

16. Collins GS, Moons KG. Comparing risk prediction models. BMJ 2012; 344: e3186. 379

17. Savvidou M, Nelson SM, Makgoba M, et al. First-trimester prediction of gestational 380

diabetes mellitus: examining the potential of combining maternal characteristics and 381

laboratory measures. Diabetes 2010; 59: 3017–22. 382

18. Popova P V, Grineva EN, Gerasimov AS, et al. The new combination of risk factors 383

determining a high risk of gestational diabetes mellitus. Minerva Endocrinol 2014; 40: 384

239–47. 385

19. Caliskan E, Kayikcioglu F, Ozturk N, Koc S, Haberal A. A population-based risk factor 386

scoring will decrease unnecessary testing for the diagnosis of gestational diabetes 387

mellitus. Acta Obstet Gynecol Scand 2004; 83: 524–530. 388

20. Eleftheriades M, Papastefanou I, Lambrinoudaki I, et al. Elevated placental growth 389

factor concentrations at 11-14 weeks of gestation to predict gestational diabetes 390

mellitus. Metabolism 2014; 63: 1419–25. 391

21. Gabbay-Benziv R, Doyle LE, Blitzer M, Baschat AA. First trimester prediction of 392

maternal glycemic status. J Perinat Med 2014; 43: 283–9. 393

22. Nanda S, Savvidou M, Syngelaki A, Akolekar R, Nicolaides KH. Prediction of gestational 394

diabetes mellitus by maternal factors and biomarkers at 11 to 13 weeks. Prenat Diagn 395

2011; 31: 135–141. 396

23. Naylor CD, Sermer M, Chen E, Farine D. Selective screening for gestational diabetes 397

mellitus. N Engl J Med 1997; 337: 1591–1596. 398

24. Pintaudi B, Vieste GD, Corrado F, et al. Improvement of selective screening strategy 399

for gestational diabetes through a more accurate definition of high-risk groups. Eur J 400

Endocrinol 2014; 170: 87–93. 401

25. Savona-Ventura C, Vassallo J, Marre M, Karamanos BG. A composite risk assessment 402

model to screen for gestational diabetes mellitus among Mediterranean women. Int J 403

Gynaecol Obstet 2013; 120: 240–4. 404

26. Syngelaki A, Bredaki FE, Vaikousi E, Maiz N, Nicolaides KH. Body mass index at 11-13 405

weeks’ gestation and pregnancy complications. Fetal Diagn Ther 2011; 30: 250–65. 406

27. Tran TS, Hirst JE, Do MAT, Morris JM, Jeffery HE. A clinical model for early predicting a 407

vietnamese woman at risk of gestational diabetes mellitus. J Paediatr Child Health 408

2012; 48: 53. 409

28. Van Leeuwen M, Opmeer BC, Zweers EJ, et al. Estimating the risk of gestational 410

diabetes mellitus: a clinical prediction model based on patient characteristics and 411

medical history. BJOG 2010; 117: 69–75. 412

29. Collins GS, Reitsma JB, Altman DG, Moons KGM. Transparent Reporting of a 413

multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): The 414

TRIPOD Statement. Ann Intern Med 2015; 162: 55. 415

30. Correction: Transparent Reporting of a multivariable prediction model for Individual 416

Prognosis Or Diagnosis (TRIPOD): The TRIPOD Statement. Ann Intern Med 2015; 162: 417

600. 418

Page 20 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

23 April 2016 20

31. World Health Organization. Definition, diagnosis and classification of diabetes mellitus 419

and its complications. Part 1: Diagnosis and classification of diabetes mellitus. 1999. 420

32. Nederlandse Diabetes Federatie (Dutch Diabetes Federation). NDF Zorgstandaard 421

Diabeteshttp://www.zorgstandaarddiabetes.nl/wp-content/uploads/2015/08/NDF-422

Zorgstandaard-zwangerschapsdiabetes-2015.pdf (2015). 423

33. Visser GHA, Eilers PHC, Elferink-Stinkens PM, Merkus HMWM, Wit JM. New Dutch 424

reference curves for birthweight by gestational age. Early Hum Dev 2009; 85: 737–44. 425

34. Moons KGM, de Groot JAH, Bouwmeester W, et al. Critical appraisal and data 426

extraction for systematic reviews of prediction modelling studies: the CHARMS 427

checklist. PLoS Med 2014; 11: e1001744. 428

35. Donders ART, van der Heijden GJMG, Stijnen T, Moons KGM. Review: a gentle 429

introduction to imputation of missing values. J Clin Epidemiol 2006; 59: 1087–91. 430

36. Steyerberg E, Borsboom G, van Houwelingen, HC; Eijkemans M, Habbema J. Validation 431

and updating of predictive logistic regression models: a study on sample size and 432

shrinkage. Stat Med 2004; 23: 2567–2586. 433

37. Toll, DB; Janssen, KJ; Vergouwe, Y; Moons K. Validation, updating and impact of 434

clinical prediction rules: a review. J Clin Epidemiol 2008; 61: 1085–1094. 435

38. Janssen KJM, Vergouwe Y, Kalkman CJ, Grobbee DE, Moons KGM. A simple method to 436

adjust clinical prediction models to local circumstances. Can J Anaesth = J Can 437

d’anesthésie 2009; 56: 194–201. 438

39. Harrel F. Regression modeling strategies: with applications to linear models, logistic 439

regression, and survival analysis. New York: Springer-Verlag, 2001. 440

40. Teh WT, Teede HJ, Paul E, et al. Risk factors for gestational diabetes mellitus: 441

Implications for the application of screening guidelines. Aust New Zeal J Obstet 442

Gynaecol 2011; 51: 26–30. 443

41. Steyerberg EW, Vickers AJ, Cook NR, et al. Assessing the performance of prediction 444

models: a framework for traditional and novel measures. Epidemiology; 21: 128–38. 445

42. American Diabetes Association. Standards of medical care in diabetes--2011. Diabetes 446

Care 2011; 34 Suppl 1: S11–61. 447

43. Nankervis A, McIntyre H, Moses R, et al. ADIPS Consensus Guidelines for the Testing 448

and Diagnosis of Gestational Diabetes Mellitus in Australia. Australas Diabetes 449

Pregnancy Soc 2008; 1–8. 450

44. Bain E, Crane M, Tieu J, et al. Diet and exercise interventions for preventing 451

gestational diabetes mellitus. Cochrane database Syst Rev 2015; 4: CD010443. 452

45. Sanabria-Martínez G, García-Hermoso A, Poyatos-León R, et al. Effectiveness of 453

physical activity interventions on preventing gestational diabetes mellitus and 454

excessive maternal weight gain: a meta-analysis. BJOG 2015; 122: 1167–74. 455

46. Ainuddin JA, Kazi S, Aftab S, Kamran A. Metformin for preventing gestational diabetes 456

in women with polycystic ovarian syndrome. J Coll Physicians Surg Pak 2015; 25: 237–457

41. 458

47. Zhuo Z, Wang A, Yu H. Effect of metformin intervention during pregnancy on the 459

Page 21 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

23 April 2016 21

gestational diabetes mellitus in women with polycystic ovary syndrome: a systematic 460

review and meta-analysis. J Diabetes Res 2014; 2014: 381231. 461

462

Page 22 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

23 April 2016 22

Tables and figures 463

Figure 1. Flow chart of participants of the RESPECT cohort 464

Figure 2. Calibration plots original prognostic models 465

Figure 3. Calibration plots recalibrated prognostic models 466

Figure 4. Calibration plots of recalibrated prognostic models in nulliparous subgroup 467

Table 1. Summary of predictors per model 468

Table 2. Baseline characteristics stratified by variables that were available for imputation 469

Table 3. Predictive performance of prognostic models for gestational diabetes on the 470

RESPECT cohort 471

Appendix A. Full equations for GDM risk prediction models as applied in the validation 472

cohort 473

Appendix B. Description of predictors 474

Appendix C. Transparency declaration 475

Appendix D. TRIPOD checklist 476

Page 23 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

23 April 2016 23

Table 1. Summary of predictors per model 477

Predictors Ca

lisk

an

20

04

Ele

fth

eri

ad

es

20

14

Ga

bb

ay

-Be

nzi

v 2

01

4

Na

nd

a 2

01

0

Na

ylo

r 1

99

7

Pin

tau

di

20

14

Sa

vo

na

-Ve

ntu

ra 2

01

3

Sh

ira

zia

n 2

00

9

Sy

ng

ela

ki

20

11

Te

ed

e 2

01

1

Tra

n 2

01

3

Va

n L

ee

uw

en

20

10

To

tal

Maternal age X X X X X X X X X X 10

Weight X 1

BMI, pre-pregnancy X 1

BMI X X X X X X X X X 9

Blood pressure X X 2

Hx of GDM X X X X 4

Family hx of DM X X X X X 5

Hx of chronic hypertension X X 2

Ethnicity X X X X X X 6

Parity X X 2

Poor obstetric outcome X 1

Hx of macrosomia X X X X 4

Method of conception X 1

Smoking X 1

Glucose X X 2

Total 5 2 5 5 3 4 3 3 8 5 3 5

BMI, body mass index; hx; history, GDM, gestational diabetes; DM, diabetes mellitus 1st

or 2nd

degree. 478 All predictors are measured in the first trimester at the initial prenatal visit, unless otherwise specified. 479

Page 24 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

23 April 2016 24

Table 2. Baseline characteristics stratified by variables that were available for imputation 480

Characteristic

Missing

n(%)

Complete cases

(n =2603)

At least one

missing value

(n=1120) p value

Overall

RESPECT cohort

(n=3723)

Age, yrs 168 (4.5%) 30.9(4.34) 30.7(3.91) 0.286 30.8 (4.2)

BMI pre-pregnancy,

kg/m2

46 (1.2%) 23.3 [21.2, 26.3] 23.0 [20.9, 25.9] 0.013 23.2 [21.1, 26.2]

BMI, kg/m2 182 (4.8%) 23.8 [21.6, 26.9] 23.4 [21.3, 26.3] 0.010 23.7 [21.5, 26.7]

Systolic BP, mmHg 65 (1.7%) 114 (12) 115 (12) 0.480 115 (12)

Diastolic BP, mmHg 64 (1.7%) 67( 8) 67 (8) 0.092 67 (8)

Glucose, mmol/L 171 (4.5%) 4.7 [4.3, 5.1] 4.7 [4.4, 5.1] 0.343 4.7 [4.4, 5.1]

Ethnicity,

- White

- African

- Asian

- Mixed

- Other

732 (19.7%)

1665 (89.0%)

17 (0.9%)

30 (1.6%)

44 (2.4%)

115 (4.4%)

1066 (95.2%)

2 (0.2%)

11 (1.0%)

15 (1.3%)

26 (2.3%)

<0.001*

3387 (91.0%)

30 (0.8%)

53 (1.4%)

77 (2.1%)

176 (4.7%)

Education,

- Low

- Middle

- High

223 (6.0%)

198 (7.6%)

825 (31.7%)

1357 (57.0%)

52 (4.6%)

362 (32.3%)

706 (63.0%)

0.004*

270 (7.3%)

1273 (34.29%)

2180 (58.6%)

Smoking during

pregnancy

0 (0%) 258 (9.9%) 73 (6.5%) 0.001* 334 (9.0%)

History of chronic

hypertension

1 (0.0%) 43 (1.7%) 14 (1.2%) 0.440 57 (1.5%)

Family history of DM 1 (0.0%) 389 (15.0%) 154 (13.8%) 0.368 543 (14.6%)

Method of conception

- Spontaneous

- Ovulation drugs

- IVF

30 (0.8%)

2396 (93.1%)

61 (2.4%)

82 (3.2%)

1033 (92.2%)

38 (3.4%)

28 (2.5%)

0.199

3429 (92.9%)

99 (2.7%)

110 (3.0%)

Nulliparous 4 (0.0%) 1143 (44.0%) 509 (45.4%) 0.429 1655 (44.5%)

History of GDM 0 (0.0%) 47 (1.8%) 12 (1.1%) 0.133 59 (1.6%)

History of macrosomia

(>90th

percentile)

0 (0.0%) 146 (5.6%) 84 (7.5%) 0.034* 230 (6.2%)

Recurrent

miscarriages (≥2)

4 (0.0%) 173 (6.7%) 59 (5.3%) 0.125 232 (6.2%)

History of foetal death 0 (0.0%) 58 (2.2%) 16 (1.4%) 0.140 74 (2.0%)

GDM in pregnancy

- Insulin dependent

263 (7.0%) 116 (5.0%)

20 (0.8%)

53 (4.7%)

13 (1.2%)

0.839

0.327

181 (4.9%)

33 (0.9%)

Gestational age at

delivery, days

342 (9.2%) 280 [273, 285] 280 [274, 286] 0.357 280 [273, 285]

Sex, male 358 (9.6%) 1154 (50.7%) 569 (52.3%) 0.400 1902 (51.1%)

Birth weight, g

- percentile

- >90th

percentile

372 (10.0%) 3504

[3200, 3860]

55 [30, 77]

256 (12.0%)

3540

[3216, 3880]

57 [32, 80]

140 (13.2%)

0.127

0.066

0.367

3520

[3190, 3880]

55 [30, 79]

494 (13.3%)

yrs, years; BMI, body mass index; BP, blood pressure; DM, diabetes mellitus; IVF, in vitro fertilization; GDM, 481 gestational diabetes; Data are n, n(%), mean (SD), or median [IQR]. The column ‘overall RESPECT cohort’ 482 includes imputed data for those with missing values. * Significant at the P < 0.005 level. 483

Page 25 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

23 April 2016 25

Table 3. Predictive performance of prognostic models for gestational diabetes on the 484

RESPECT cohort 485

Prediction model

C statistic

Development

C statistic

Recalibrated

C statistic

Nulliparous

Caliskan

2004

NR 0.73

[0.69-0.76]

0.74

[0.68-0.80]

Eleftheriades

2014

0.73

[0.65-0.81]

0.70

[0.65-0.74]

0.73

[0.66-0.79]

Gabbay-Benziv

2014

0.81

[0.77-0.87]

0.75

[0.71-0.79]

0.72

[0.66-0.79]

Nanda

2011

0.79

[0.76-0.82]

0.78

[0.74-0.82]

0.76

[0.70-0.83]

Naylor

1997

0.69

NR

0.72

[0.69-0.76]

0.71

[0.65-0.77]

Pintaudi

2014

NR 0.72

[0.68-0.75]

0.73

[0.67-0.79]

Savona-Ventura

2013

0.89

[0.86-0.91]

0.68

[0.64-0.72]

0.72

[0.65-0.78]

Shirazian

2009

NR 0.71

[0.67-.75]

0.73

[0.66-0.79]

Syngelaki

2011

NR 0.71

[0.66-0.75]

0.76

[0.66-0.79]

Teede

2011

0.70

NR

0.77

[0.73-0.81]

0.76

[0.69-0.82]

Tran

2013

0.63

[0.60-0.65]

0.67

[0.63-0.72]

0.69

[0.63-0.76]

Van Leeuwen

2010

0.77

[0.69-0.85]

0.75

[0.71-0.78]

0.77

[0.71-0.84]

The C statistic of ‘development’ shows the C statistics as reported in the original publication if available. The C 486 statistics ‘recalibrated’ shows the C statistics per model, recalibrated to the RESPECT cohort. The C statistics 487 ‘Nulliparous’ shows the C statistic per model when applied to a subgroup of only nulliparous. 488 Data are presented in mean [95% confidence interval]. NR = not reported. 489

Page 26 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

Figure 1. Flow chart of participants of the RESPECT cohort 3,723 women were included for 145x80mm (300 x 300 DPI)

Page 27 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

Figure 2. Calibration plots original prognostic models

Three original publications pr

176x72mm (300 x 300 DPI)

Page 28 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

Figure 3. Calibration plots recalibrated prognostic models

Calibration plots were also dr

176x291mm (300 x 300 DPI)

Page 29 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

Figure 4. Calibration plots of recalibrated prognostic models in nulliparous subgroup Calibration of the prognostic 176x291mm (300 x 300 DPI)

Page 30 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nlyAppendix A. Full model equations for gestational diabetes as applied to the RESPECT cohort

Caliskan 2004

The probability of developing gestational diabetes was calculated as:

X = 1 (if adverse outcome(i.e. recurrent (≥2) abortions & previous IUFD)) + 1 (if age ≥25 yrs)

+ 1 (if BMI ≥25 kg/m2) + 1 (if family history of DM, first degree) + 1 (if parous with

previous LGA above 90th

percentile)

Eleftheriades 2014

The probability of developing gestational diabetes was calculated as: eX/(1+e

X), where

X = α + 0.058 (weight, kg) + 0.182 (age, yrs)

Gabbay-Benziv 2014

The probability of developing gestational diabetes was calculated as: 1/(1+e-(X)), where

X = -11.569 + 0.064 (age, yrs) + 0 (if white race) + 2.026 (if Asian race) + 0.083 (if African

race) + 1.661 (if other nonwhite race) + 2.144 (history of GDM) + 0.034 (systolic

blood pressure, mmHg) + 0.082 (BMI, kg/m2)

Nanda 2011


X), where

X = α + 0.058 (age, yrs) + 0.113 (BMI, kg/m2) + 0 (if Caucasian ethnicity) + 0.888 (if Asian

ethnicity) + 0 (nulliparous) + 3.723 (if parous with previous GDM) + 0.67 (parous with

previous LGA above 90th

percentile)

Naylor 1997


X), where

X = α + 0 (age ≤30 yrs) + 0 (age 31-34 yrs) + 0.47 (age ≥35 yrs) + 0 (BMI ≤22.0 kg/m2) + 0.588

(BMI 22.1-25.0 kg/m2) + 1.163 (BMI ≥25.1 kg/m

2) + 0 (if white race) - 0.357 (if black

Page 31 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nlyrace) + 1.569 (if Asian race) + 0.47 (if other race)

Pintaudi 2014


X), where

X = α + 0 (class 4) + 1.361 (class 3) + 1.856 (class 2) + 3.091 (class 1) + 1.281 (previous LGA

with birth weight ≥4500 gr) + 0.588 (if family history of DM, first degree)

Class 1: random glucose >5.1 mmol/L

Class 2: random glucose >4.4-≤ 5.1 mmol/L & pre-pregnancy BMI >24.4 kg/m2

Class 3: random glucose >4.4-≤ 5.1 mmol/L & pre-pregnancy BMI ≤24.4 kg/m2

Class 4: random glucose ≤4.4 mmol/L

Savona-Ventura 2013


X), where

X = -4.144 + 3.142 (if glucose >5.0 mmol/L) + 0.758 (if age ≥30 yrs) + 0.543 (if diastolic blood

pressure ≥80 mmHg)

Shirazian 2009


X), where

X = α + 0 (if age ≤24 yrs) + 0.512 (if age 25-29 yrs) + 1.515 (if age ≥30 yrs) + 0 (if BMI pre-

pregnancy ≤24.9 kg/m2) + 0.513 (if BMI pre-pregnancy 25.0-29.9 kg/m

2) + 0.892 (if

BMI pre-pregnancy ≥30.0 kg/m2) + 0.842 (if family history of DM, first degree)

Syngelaki 2011


X), where

X = α + 0.014 (BMI, kg/m2) + 0.068 (age, yrs) + 0 (if Caucasian race) + 0.344 (if African race) +

1.050 (if Asian race) + 0.174 (if mixed race) + 0 (if spontaneous conception) + 0.432

(if conception with ovulation drugs) + 0.312 (if conception via IVF) + 0.020 (if

Page 32 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nlysmoking) – 0.010 (if chronic hypertension) + 0 (if nulliparous) – 0.211 (if parous

without previous LGA above 95th

percentile) + 0.663 (if parous with previous LGA

above 95th

percentile)

Teede 2011


X), where

X = α + 0 (if age <25 yrs) + 0.92 (if age 25-29 yrs) + 1.22 (if age 30-34 yrs) + 1.69 (if age 35-39

yrs) + 1.95 (if age ≥40 yrs) + 0 (if BMI <20 kg/m2) + 0.53 (if BMI 20.0-24.9 kg/m

2) +

0.69 (if BMI 25.0-26.9 kg/m2) + 0.83 (if BMI 27.0-29.9 kg/m

2) + 1.28 (if BMI 30.0-34.9

kg/m2) + 1.82 (if BMI ≥35.0 kg/m

2) + 1.31 (if Asian race) + 0.06 (if African race) + 0.37

(if other race) + 0.53 (if family history of DM, first degree) + 2.39 (if history of GDM)

Tran 2013


X), where

X = α + 0.351 (age, yrs) + 0.131 (BMI, kg/m2)

Van Leeuwen 2010


X), where

X = -6.1 + 0.83 (if non-Caucasian race) + 0.57 (if family history of DM, first degree) – 0.67 (if

parous without history of GDM) + 0.5 (if parous with history of GDM) + 0.13 (BMI in

kg/m2 with BMI <22 transformed to 22, if > 30 transformed to 30)

IUFD, intra uterine fetal death; yrs, years; BMI, body mass index; DM, diabetes mellitus; LGA,

large for gestational age; kg, kilograms; GDM, gestational diabetes; gr, grams; IVF, in vitro

fertilization

Page 33 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nlyAppendix B. Description of predictors

Predictor measurement in validation cohort

Gestational age was calculated based on the first-trimester crown rump length

measurement at ultrasound examination (dating scan) using the formula of Robinson and

Fleming.1 Anthropometric variables were all measured at initial visit in the first trimester;

maternal body weight was measured in kilograms, height was measured in centimetres.

Body mass index (BMI) was calculated as weight (kilograms) divided by the squared height

(meters) of the subject. Blood pressure was measured in mmHg following standard

procedures.2 Mean arterial pressure (MAP) was calculated as (1/3 * systolic blood pressure)

+ (2/3 * diastolic blood pressure). A random glucose was measured in mmol/L. Maternal

characteristics as well as medical and obstetrical history were obtained through a self-

administered questionnaire. The following variables were relevant for this study: age (years),

ethnicity (Caucasian, African, Hindustani, Moroccan, Turkish, Middle Eastern, Asian, other

western, other nonwestern, and mixed), level of education (low, primary education or lower

level; middle, secondary education; high, tertiary education or higher level), cigarette

smoking during pregnancy (yes or no), method of conception (spontaneous, use of ovulation

drugs or in vitro fertilization), history of chronic hypertension (yes or no) and parity

(nulliparous with no previous pregnancy beyond 16 weeks or parous women), family history

of DM (first degree relative, yes or no), birth weight (grams) and birth weight percentile of

infants of previous pregnancies (based on national reference curves adjusted for parity, GA,

sex and ethnicity)3, previous pregnancies with GDM (yes or no), two or more spontaneous

abortions (yes or no), prior intra-uterine fetal death after 20 weeks of gestation (yes or no).

Page 34 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nlyAdjustments for external validation

Several continuous variables were transformed into categorical variables, according to

definitions in the original prediction model. This was the case for: age (Caliskan 2004, Naylor

1997, Shirazian 2009, Teede 2011), BMI (Caliskan 2004, Naylor 1997, Shirazian 2009, Teede

2011), and birth weight percentile of prior infants (Caliskan 2004, Syngelaki 2011). For one

model (Caliskan 2004) the cut-off for birth weight percentile of prior infants was not

specified nor provided on request. The most frequent used definition was applied (≥90th

percentile).

For family history of DM different definitions were used: one model (Shirazian 2009) defined

it as a relative with DM type II, degree not specified. A second model (Teede 2011) specified

a first degree relative, type of diabetes not specified. Another model considered family

history of DM positive if a 1st or 2nd degree relative has type I of type II DM (Van Leeuwen

2011). For the RESPECT-study participants family history of first degree relatives was

administered, there was no distinction made between type of DM. Therefore, the predictor

family history of DM was considered positive in all models if a first degree relative has any

type of DM.

Ethnicity was divided into ten subgroups in the original RESPECT-study. Most studies

(Gabbay-Benziv 2014, Naylor 1997, Syngelaki 2011) divided ethnicity into four groups:

White, Asian, Black, other. Ethnicity was recoded into these categories according to the

prediction models. Two studies (Nanda 2011 & Teede 2011) distinguished different Asian

types, for the RESPECT-study this was not possible. The most frequent Asian type was

Chinese, therefore we choose for east Asian of Chinese Asian predictor coefficients. Other

Asian type predictor coefficients were thereby excluded from the model.

Page 35 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nlyOne model (Teede 2011) included ‘poor obstetric outcome’ in the prediction model. This

variable was not further specified, and showed no significant contribution in the original

prediction model. Therefore this predictor was excluded from the model.

References

1. Robinson HP, Fleming JE. A critical evaluation of sonar ‘crown-rump length’ measurements. Br J Obstet Gynaecol 1975; 82: 702–10.

2. De Boer J, Zeeman K, Verhoeven C. Hypertensive disorders in pregnancy, labour and post partum period. 2011.

3. Visser GHA, Eilers PHC, Elferink-Stinkens PM, Merkus HMWM, Wit JM. New Dutch reference curves for birthweight by gestational age. Early Hum Dev 2009; 85: 737–44.

Page 36 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nlyAppendix C. Transparency declaration

Corresponding author

The corresponding author has the right to grant on behalf of all authors and does grant on

behalf of all authors, an exclusive licence on a worldwide basis to the BMJ Publishing Group

Ltd to permit this article (if accepted) to be published in BMJ editions and any other BMJPGL

products and sublicences such use and exploit all subsidiary rights, as set out in our licence.

Declaration of competing interests All authors have completed the Unified Competing

Interest form at www.icmje.org/coi_disclosure.pdf (available on request from the

corresponding author) and declare no support from any organization for the submitted work; no

relationships with companies that might have an interest in the submitted work in the

previous 3 years; no other relationships or activities that could appear to have influenced the

submitted work.

Details of contributors MPHK, AK, AF, KGMM and the RESPECT study group had the original

idea for the study and were involved in writing the original study protocol. The RESPECT

study group and MLdR were involved in data collection. CAN and MLdR performed data

analysis. MLdR, CAN, and MPHK wrote the first draft of the manuscript, which was

subsequently revised by AF, AK and KGMM. All authors participated in the final approval of

the manuscript. MPHK and AF are the guarantors of this study.

Page 37 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nlyStatement of access to all of the data All authors had full access to all of the data (including

statistical reports and tables) in the study and take responsibility for the integrity of the data

and the accuracy of the data analysis.

Transparency declaration The lead author affirms that the manuscript is an honest,

accurate, and transparent account of the study being reported; that no important aspects of

the study have been omitted; and that any discrepancies from the study as planned (and, if

relevant, registered) have been explained.

Ethical approval This study was approved by the medical ethics committee of the University

Medical Center Utrecht (protocol number 12-432/C) and written informed consent was

obtained from all participants.

Details of funding This study has been conducted with the support of The Netherlands

Organization for Health Research and Development (project nr 50-50200-98-060). The

funding source no role in the design, conduct, analyses, or reporting of the study or in the

decision to submit the manuscript for publication.

Data sharing statement Data sharing: patient level data and full dataset and technical

appendix and statistical code are available from the corresponding author. Informed consent

was not obtained but the presented data are anonymized and the risk of identification is

low.

Page 38 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

Appendix D. TRIPOD Checklist

Section/Topic Item Checklist Item Page

Title and abstract

Title 1 D;V Identify the study as developing and/or validating a multivariable prediction model, the target population, and the outcome to be predicted.

1

Abstract 2 D;V Provide a summary of objectives, study design, setting, participants, sample size, predictors, outcome, statistical analysis, results, and conclusions.

4

Introduction

Background and objectives

3a D;V Explain the medical context (including whether diagnostic or prognostic) and rationale for developing or validating the multivariable prediction model, including references to existing models.

6

3b D;V Specify the objectives, including whether the study describes the development or validation of the model or both.

7

Methods

Source of data

4a D;V Describe the study design or source of data (e.g., randomized trial, cohort, or registry data), separately for the development and validation data sets, if applicable.

8+9

4b D;V Specify the key study dates, including start of accrual; end of accrual; and, if applicable, end of follow-up.

9+11

Participants

5a D;V Specify key elements of the study setting (e.g., primary care, secondary care, general population) including number and location of centres.

9

5b D;V Describe eligibility criteria for participants. 9

5c D;V Give details of treatments received, if relevant. NA

Outcome 6a D;V

Clearly define the outcome that is predicted by the prediction model, including how and when assessed.

10

6b D;V Report any actions to blind assessment of the outcome to be predicted. NA

Predictors

7a D;V Clearly define all predictors used in developing or validating the multivariable prediction model, including how and when they were measured.

9

7b D;V Report any actions to blind assessment of predictors for the outcome and other predictors.

9

Sample size 8 D;V Explain how the study size was arrived at. 11

Missing data 9 D;V Describe how missing data were handled (e.g., complete-case analysis, single imputation, multiple imputation) with details of any imputation method.

11

Statistical analysis methods

10a D Describe how predictors were handled in the analyses. NA

10b D Specify type of model, all model-building procedures (including any predictor selection), and method for internal validation.

NA

10c V For validation, describe how the predictions were calculated. App A +B

10d D;V Specify all measures used to assess model performance and, if relevant, to compare multiple models.

12

10e V Describe any model updating (e.g., recalibration) arising from the validation, if done. 11 + App B

Risk groups 11 D;V Provide details on how risk groups were created, if done. NA

Development vs. validation

12 V For validation, identify any differences from the development data in setting, eligibility criteria, outcome, and predictors.

App A

Results

Participants

13a D;V Describe the flow of participants through the study, including the number of participants with and without the outcome and, if applicable, a summary of the follow-up time. A diagram may be helpful.

13

13b D;V Describe the characteristics of the participants (basic demographics, clinical features, available predictors), including the number of participants with missing data for predictors and outcome.

Table 2

13c V For validation, show a comparison with the development data of the distribution of important variables (demographics, predictors and outcome).

NA

Model development

14a D Specify the number of participants and outcome events in each analysis. NA

14b D If done, report the unadjusted association between each candidate predictor and outcome.

NA

Model specification

15a D Present the full prediction model to allow predictions for individuals (i.e., all regression coefficients, and model intercept or baseline survival at a given time point).

NA

15b D Explain how to the use the prediction model. NA

Model performance

16 D;V Report performance measures (with CIs) for the prediction model. Table 3

Model-updating 17 V If done, report the results from any model updating (i.e., model specification, model performance).

Table 3

Discussion

Limitations 18 D;V Discuss any limitations of the study (such as nonrepresentative sample, few events per predictor, missing data).

16

Interpretation 19a V

For validation, discuss the results with reference to performance in the development data, and any other validation data.

16

19b D;V Give an overall interpretation of the results, considering objectives, limitations, results from similar studies, and other relevant evidence.

15

Implications 20 D;V Discuss the potential clinical use of the model and implications for future research. 17

Other information

Supplementary information

21 D;V Provide information about the availability of supplementary resources, such as study protocol, Web calculator, and data sets.

App A-C

Funding 22 D;V Give the source of funding and the role of the funders for the present study. App C

Page 39 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960


nly

Appendix D. TRIPOD Checklist

*Items relevant only to the development of a prediction model are denoted by D, items relating solely to a validation of a prediction model are

denoted by V, and items relating to both are denoted D;V. We recommend using the TRIPOD Checklist in conjunction with the TRIPOD

Explanation and Elaboration document.

Page 40 of 39


BMJ

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960

bmj...13 care, university medical centre utrecht, str. 6.131, po box 85500, utrecht, the netherlands...

Documents