the control of allergic rhinitis and asthma test...
TRANSCRIPT
Eindverslag Stage wetenschap
The Control of Allergic Rhinitis and Asthma
Test: validation of the Dutch version
Sander van der Leeuw
s1428098
Facultair begeleider: Prof. T. van der Molen
Dagelijks begeleider: Dr. B.M.J. Flokstra
Rijksuniversiteit Groningen, UMCG, Afdeling Huisartsgeneeskunde
1
The Control of Allergic Rhinitis and Asthma Test: validation
of the Dutch version
S van der Leeuw
Rijksuniversiteit Groningen, Universitair Medisch Centrum Groningen, Afdeling Huisartsgeneeskunde en GRIAC research instituut, Groningen
Background: Simultaneous treatment of allergic rhinitis and asthma has been
recommended by the Allergic Rhinitis and its Impact on Asthma (ARIA) guidelines.
The Portuguese Control of Allergic Rhinitis and Asthma Test (CARAT) was therefore
developed to monitor the control of both asthma and allergic rhinitis in primary and
secondary care. This questionnaire was translated to Dutch.
Objectives: The primary objectives of this study were to evaluate the internal
consistency, construct validity and longitudinal validity of the Dutch CARAT.
Furthermore, test-retest reliability and Minimal Clinically Important Difference (MCID)
were investigated.
Methods: The Dutch CARAT study is a longitudinal study consisting of three
measurements with one month intervals. 176 patients diagnosed with asthma and/or
rhinitis from three primary care practices and three secondary care centers were
approached. Cronbach’s alpha was used to evaluate internal consistency. CARAT
scores (ranging from 0-30) were compared to the Asthma Control Questionnaire
(ACQ) and VAS scores concerning airway symptoms. Spearman’s correlation
coefficients were used to determine construct and longitudinal validity. For the
longitudinal analysis delta scores (t2-t1 and t3-t2) of the CARAT, ACQ and VAS were
used. Test-retest reliability and Minimal Clinically Important Difference (MCID) were
evaluated using Global Rating of Change (GRC) scores to define categories
according to change in symptoms over time.
Results: A total of 93 patients were included. The mean CARAT score for all patients
was 19.4 (SD 6.80). Cronbach’s alpha for the CARAT was 0.82. Correlation
coefficients between CARAT and the ACQ and VAS questions ranged from
0.64(p<0.01) to 0.76(p<0.01). Longitudinally, correlation coefficients between delta
CARAT scores and delta ACQ and VAS ranged from 0.41 to 0.67 (p<0.01) for both
one month intervals. Calculations for test-retest reliability showed correlation
coefficients of 0.74(p<0.01) and 0.78(p<0.01) between CARAT scores of patients in
the ‘no change’-category according to GRC. The MCID for the CARAT was 3.50.
Conclusion: The Dutch CARAT has high internal consistency and good construct
validity and is therefore comparable to findings according to the Portuguese CARAT.
The Dutch CARAT also proves to be longitudinally valid, with good test-retest
reliability. The determination of MCID is highly beneficial for interpreting CARAT
scores. Therefore, the CARAT is a valid and reliable tool for use in clinical research
and practice to monitor asthma and allergic rhinitis symptoms simultaneously.
2
Table of contents
Background………………………………. ..3
Methods…………………………………... ..5
Results……………………………………. 10
Discussion……………………………….. 16
Conclusion………...……………………... 18
References……..………………………… 19
Appendix……….………………………… 21
3
Background
Asthma and allergic rhinitis are very common diseases that represent a global health
problem. Epidemiologic studies consistently show that asthma and allergic rhinitis
frequently co-exist (1). This has led to the concept that asthma and allergic rhinitis
are strongly related disorders with symptoms occurring in the upper airways (allergic
rhinitis) or lower airways (asthma). A number of studies have shown that 70-90% of
patients with asthma also have allergic rhinitis and 40-50% of patients with allergic
rhinitis also have asthma. Furthermore, there is a probable association between the
severity of asthma and allergic rhinitis (2-5).
The quality of asthma and allergic rhinitis control is of great influence on patients’
quality of life (6). Asthma and allergic rhinitis symptoms have shown to have a
negative influence on social life, school performance, and work productivity (1).
Patients who suffer from severe forms of disease are notably affected (7). Also,
allergic rhinitis symptoms have a subversive effect on academic performance. The
expenses caused by subjects with allergic rhinitis are substantial (8).
Most patients with asthma and/or allergic rhinitis are treated by general practitioners,
who are confronted with a variety of treatment options. Given the variation in patients,
symptoms and clinical presentation, primary care clinicians are faced with uncertainty
about the considerations regarding asthma and allergic rhinitis management (9,10).
Although 70-90% of patients with asthma also suffer from allergic rhinitis, symptoms
of asthma are generally only measured by questionnaires specific for asthma and not
for allergic rhinitis (11-14). General practitioners are therefore often not aware of
allergic rhinitis symptoms in their patients although their allergic rhinitis symptoms
might have a large impact on asthma control. This results in a large number of
patients who do not receive appropriate care despite the availability of effective
treatment options.
Clinical practice guidelines for the management of allergic rhinitis have been
developed over the past 15 years and have improved the care of this group of
patients (9). The first in the field of these evidence-based guidelines were the Allergic
Rhinitis and its Impact on Asthma (ARIA) guidelines. These guidelines were
developed in collaboration with specialists in allergy, primary care physicians, and
patient representatives from the European Federation of Allergy and Airways
Diseases Patients Associations (EFA)(15).
The ARIA guidelines recommend the optimal control of both asthma and allergic
rhinitis airway disease as the primary goal of their treatment (1,16). A combined
approach to the management of upper and lower airway disease has been
extensively proposed (1,15-17). To assess the effects on control of treatment,
validated questionnaires have been proposed as key instruments (18-20).
4
The Control of Allergic Rhinitis and Asthma Test (CARAT) questionnaire was created
to evaluate the effect of treatment for both asthma and allergic rhinitis (21). It is a
self-administered questionnaire that consists of 10 items monitoring upper and lower
airway symptoms and the use of medication. The CARAT questionnaire has been
validated to assess disease control of both asthma and allergic rhinitis in the
Portuguese population. Also, statistically redundant questions were removed,
shortening the CARAT questionnaire from 17 to 10 items, thus reducing the
completion time (6,21). The CARAT questionnaire translation is currently in different
steps in several languages including English, French, Greek, Spanish, Italian,
Turkish, Spanish, Finnish, Swedish, Norwegian, and Danish. Translation and cultural
adaptation of the CARAT questionnaire in the Netherlands was finished in November
2011. The Dutch CARAT study is part of the research focus “COPD, asthma and
allergy in general practice”, at the UMCG department of general practice. There is no
experience with the CARAT questionnaire in the Netherlands.
The objective of the current study is to evaluate the internal consistency and cross-
sectional validity for the Dutch version of the CARAT, as is done for the Portuguese
version of the CARAT. In addition, longitudinal validity, test-retest reliability and
Minimal Clinically important difference of the translated CARAT are investigated.
Another goal is to evaluate the sensitivity of the CARAT for detecting the influence of
pollen counts on asthma and allergic rhinitis symptoms.
Specific activities during internship
During my internship concerning the CARAT study I was responsible for distribution
of the questionnaire packages and the data entry. Logistic monitoring of the
questionnaires (using an Excel database) and the distribution of reminders were
further activities. Other responsibilities concerned the selection and inclusion of
patients, gathering patient characteristics from the centers in question, and creating
an SPSS database for the CARAT study. I was also involved in study design,
presentations concerning the CARAT study, performing statistical analyses and
writing a scientific paper.
5
Methods
Study design
The (CARAT study) is a longitudinal study consisting of three measurement points
with one month intervals (T1, T2, T3). The study was conducted between February
2012 and December 2012.
Participants
Patients were approached from the outpatient clinics from the department of allergy
in the UMCG (Dr. J.N.G Oude Elberink), the department of pulmonary diseases of
the Radboud University (Prof. R. Dekhuizen), and the allergy department of the
Erasmus University (Prof. R. Gerth van Wijk). Patients from primary care practices of
Dr. F.R. van Gemert in Harlingen, Dr. J.W.H. Kocks in Groningen and Dr. R.A.
Riemersma in Appingedam were also approached. From these six centers a total of
176 patients with a physician diagnosis of asthma and/or allergic rhinitis were asked
to participate. Patients from 18 to 70 years of age were included. Patients with
insufficient command of the Dutch language and patients diagnosed with dementia
were excluded from the study. Informed consent was obtained from all patients in
writing before the acquisition of baseline characteristics. House addresses of all
participants were acquired from each center for logistic purposes.
Sample size could not be calculated by power analysis, since there is no expected
difference in the process of validation. There are no general criteria for the required
sample size in a validation study of a translated questionnaire. A sample size of at
least 50-100 participants is generally recommended. After one round of
questionnaires and reminders the number of participants to be included in the study
was 92.
Baseline characteristics were obtained from the patient files in each center. A special
form was created for this purpose, including the following characteristics: age, sex,
age of onset asthma and/or allergic rhinitis, history of smoking, allergens to which
patient is allergic (clinical reactivity), specific IgE and / or skin prick test including tree
pollen, grass pollen, and other relevant allergens, lung function (FEV1%pred,
reversibility) and respiratory co-morbidity.
Materials
The CARAT questionnaire was developed in Portugal to measure the quality of
control of both asthma and allergic rhinitis (21). It was validated for the Portuguese
population in 2010. Cronbach’s alpha was 0.86 for the Portuguese CARAT. The
6
CARAT scores showed a good correlation with valid questionnaires for asthma and
allergic rhinitis symptoms. (6).Translation to Dutch was finished in February 2012.
The Dutch CARAT questionnaire* consists of 10 multiple choice questions which
refer to the four weeks prior to the date of completion. Seven questions relate to the
frequency of airway symptoms, four of which are upper airway symptoms and three
are lower airway symptoms. The other three questions concern sleep impairment,
activities limitation, and need for higher doses of medication respectively.
The first 9 questions are scored on a four point Likert scale ranging from 0 points
(almost every day) to 3 points (never), a higher score reflecting better asthma and
allergic rhinitis control. The last question concerning additional medication uses the
same scale, but it is scored in a different way: 3 points for the answers “I take no
medication” and “never”, 2 points for “less than seven days” and 0 points for “more
than seven days”. The total score is calculated by summing up the scores off all ten
questions, resulting in a range of 0-30 points. The total CARAT score and the two
separate factors of the questionnaire were distinguished in statistic analyses. Factor
1 consists of CARAT questions 1-4 and represents the questions about allergic
rhinitis symptoms. Factor 2 consists of questions 5-10 and represents the questions
about asthma symptoms (6).
The asthma control questionnaire (ACQ)* consists of six questions which are scored
on a 7 point Likert scale. These questions are based on the participants’ situation of
the last seven days. The total ACQ score is the mean score of all questions (ranging
from 0 to 6), a lower score representing better asthma control. The ACQ has been
shown to be reliable (ICC = 0.90, p<0.0001) and have strong evaluative properties
for the measurement of asthma control (18). It has also shown to have good
discriminative qualities which enable the ACQ to distinguish between patients who
have well controlled and not well controlled asthma. (22)
Three visual analog scales (VAS)* were used to assess general airway symptoms
and symptoms specific for asthma and allergic rhinitis. VAS question 1 covered all
airway symptoms, question 2 concerned lower airway symptoms and question 3
concerned higher airway symptoms. Participants were asked to mark the location on
a 10 centimeter line corresponding to the amount of symptoms they experienced in
the preceding week. Visual analog scales have shown to be highly responsive to
change during treatment when used for a global evaluation of allergic rhinitis
symptoms (23).
For the second and third measurement (T2 and T3) a global rating of change (GRC)*
question with a 15 point scale was used to monitor the participants’ subjective
experience of symptoms of asthma and allergic rhinitis, compared to the previous
measurement. The score range of this question was from -7 (no symptoms) to 7
(extreme symptoms).
* Questionnaire package T2 included in appendix
7
In addition, daily pollen-specific counts were provided by the
LUMC, measured by a pollen sampler on the roof of the 6th floor.
The filter is replaced weekly and analyzed in a laboratory, to
determine the pollen counts per m3 for every 24 hours. These daily
counts discriminate 41 different pollen species. Eight of these are
classified as allergenic, in three different categories: mildly
allergenic (four species), very allergenic (two species) and
extremely allergenic (two species). Only the eight allergenic pollen
species were used for statistical analysis.
Procedure
The CARAT study was approved by the Medical Ethical Commission (METc) in
February 2012, who deemed that permission from the commission was not required.
Questionnaire packages were distributed by mail to 176 patients with an
accompanying information letter from both clinician and researcher and an informed
consent form. Participants were requested to return the completed questionnaires
and informed consent form in a post-paid envelope. The first questionnaire
packages* were sent to the patients in May 2012.
All participants were asked to fill out a questionnaire package three times with a time
interval of a month (T1, T2, T3). The questionnaire packages included: the CARAT,
the ACQ5, the VAS on pulmonary symptoms, and questions on present medication
use and demographics. The GRC question was added to the package for T2 and T3.
Participants were requested to state the date of completion of the first questionnaire.
This date was used for administrative purposes to time the one month follow-up
questionnaires accordingly. In this way the date could be determined for each
participant individually. The same method was used for the third questionnaire.
Reminders were sent once if a participant had not returned the questionnaire within
one week after the aimed date. The number of reminders was counted to use for
descriptive purposes.
The acquisition of completed questionnaires covered a period of five months, and
lasted from May through September 2012, which covers the allergic rhinitis season.
Pollen counts were entered in a matrix in Microsoft Excel to make them available for
statistical analysis. Entering and analyzing data from the six participating centers
were centrally coordinated by the student at the department of General Practice at
the UMCG.
* Questionnaire package T2 included in appendix
8
Statistical Analyses
Statistical analyses were performed using SPSS 19.
All variables were tested for normal distribution using QQ-plots.
Internal consistency reliability of the CARAT was evaluated by calculating Cronbach’s
alpha. A Cronbach’s alpha of at least 0.70 was expected to consider the CARAT
suitable for the comparison of groups of patients. Further priori expectations were
based on the Portuguese version of the CARAT which has a Cronbach’s alpha of
0.86.
The cross-sectional validity of the CARAT was evaluated by calculating the
correlation between CARAT scores and ACQ5 as well as VAS scores. For this, we
used total CARAT scores and the scores of the separate Factors 1 (for allergic
rhinitis) and 2 (for asthma). Factor 1 of the CARAT and VAS3 represent higher
airway symptoms whereas Factor 2 of the CARAT and VAS2 represent lower airway
symptoms. VAS1 represents airway symptoms in general. VAS2+3 is obtained by
summing VAS2 and VAS3 and using it as a combined score for general airway
symptoms. A priori expectations were based on the Portuguese version of the
CARAT which showed correlation coefficients ranging from 0.6-0.8 with the ACQ and
VAS scores. Factor 1 was expected to correlate best with the VAS score concerning
higher airway symptoms, Factor 2 was expected to correlate best with the VAS score
concerning lower airway symptoms.
Longitudinal validity was evaluated by calculating the correlation between delta
scores of the CARAT and delta scores of the ACQ and VAS scores. These
correlation coefficients were calculated for intervals T1-T2 and T2-T3, subtracting the
former score from the latter. Separate factors of the CARAT were also compared to
ACQ and VAS scores.
The minimal clinically important difference was calculated using the absolute GRC
scores |GRC| measured on T2, this measurement representing the difference in
symptoms between T1 and T2. Patients were divided into four categories based on
|GRC|: no difference (0-1), minimal difference (2-3), moderate difference (4-5) and
large difference (6-7). For each of these categories the mean difference in CARAT
score between T1 and T2 was calculated. The mean difference for the ‘minimal
difference’-group was listed as MCID (24).
Test retest reliability was evaluated using the GRC scores measured on T2 and T3,
using only the patients that had scored between -1 and 1 (no difference).
Subsequently, the correlation between the CARAT scores in consecutive
measurements was calculated for T1-T2 and T2-T3 to determine test retest reliability
for both intervals. For this, Spearman correlation coefficients were used.
The discriminative properties (ability to distinguish patients at a single time point) of
the CARAT were investigated by using ACQ scores. The sample on T1 was
separated into two groups: one group containing patients with ACQ score < 1.5 (well
9
controlled), the other containing patients with ACQ score > 1.5 (not well controlled).
These scores were used because 1.5 is the cut-off point for the ACQ to distinguish
well controlled patients from patients that are not well controlled (22). The mean
CARAT score for T1 was calculated for both groups and compared using an
independent samples t-test. The a priori expectation was to find a significantly lower
mean CARAT score for the ‘ACQ not well controlled’ group, when compared to the
‘ACQ well controlled’ group.
Finally, the correlation between pollen counts during the course of the study and the
CARAT scores were investigated. The mean pollen counts in 4 weeks prior to the
date of completing the questionnaire package were calculated for each patient, for
each measuring moment (T1, T2, T3). This resulted in one number for each date,
representing the mean pollen counts for the 28 days prior to that particular date. In
this way, a mean pollen count could be used for every individual patient, depending
on the date of completing the questionnaires. To evaluate the sensitivity of the
CARAT questionnaire to changes in pollen counts, the correlation between the
differences in mean pollen counts (individually for each patient) and the change in
their total CARAT scores was calculated. This was executed for different samples of
patients depending on their diagnoses and clinical reactivity. The expectation was to
find a consistent longitudinal correlation between CARAT scores and pollen counts
during the period represented by the questionnaires.
10
Results
Of the 176 patients that where approached for the CARAT study 92 patients completed T1. Of these 92 patients, 82 patients completed T2 and of these 82 patients, 72 patients completed T3 (Table 1). A number of these patients filled out the questionnaires after receiving a reminder and these numbers are also shown in Table 1. The patient characteristics of this sample are described in Table 2. The data concerning IgE, skin prick tests and lung function were not used in statistical analysis due to inconsistent data from different centers.
Table 1. Response rates
Table 2. Patient characteristics
Table 2. Respiratory co-morbidity in patients consisted exclusively of COPD. Clinical reactivity categories were based on all available data from patient files. All patients allergic to pollen were put in the ‘Pollen’ category, irrespective of other allergies. The category ‘Other’ consisted of patients that were clinically reactive to any other allergens than pollen. Data for skin prick tests and specific IgE could not be described due to different outcome measures in each center. The percentages represent the number of patients who underwent the investigation in question.
Measurements Completed/approached Reminders sent/ completed
T1, n (%) 92/176 (53%) 17/92 (18%)
T2, n (%) 82/92 (89%) 25/82 (30%)
T3, n (%) 72/82 (88%) 38/72 (53%)
Characteristics Primary care Secondary care
Patients , n 46 46
Age in years, mean (SD) 46.6 (127) 41.3 (142)
Sex - Male (%) - Female (%)
41.3 58.7
23.4 76.6
Age of onset in years, mean (SD) - Asthma - Rhinitis
37.0 (15.1) 32.7 (11.1)
19.8 (14.5) 19.4 (10.5)
Respiratory co-morbidity, n (%) 8.7 23.4
Clinical reactivity - None (%) - Pollen(%) - Other(%) - Unknown (%)
26.1 39.1 17.4 17.4
2.1
85.1 10.6 2.1
Skin prick test, n (%) 1 (2.2%) 30 (65.2%)
Specific IgE, n (%) 2 (4.4%) 60 (32.6%)
Lung function, n (%) 41 (89.1%) 33 (71.7%)
11
The mean score of the CARAT among all patients for T1 was 19,4 with a standard deviation of 6,8. There was a minimum score of 3 (n = 1) and a maximum score of 30 (n = 4). There were differences in mean CARAT scores among gender and age categories, listed in Table 3. Table 3. Mean CARAT scores
CARAT scores Mean (SD) n
CARAT T1 19.36 (6.80) (n=92)
CARAT T1 ♀ CARAT T1 ♂
18.63 (6.59) 20.93 (7.11)
(n=63) (n=29)
CARAT T1 age percentiles
<33
34-43
44-53
>54
17.87 (7.07) 17.38 (7.64) 19.38 (7.04) 22.24 (4.88)
(n=23) (n=23) (n=23) (n=23)
CARAT T2 19.99 (7.13) (n=82)
CARAT T3 21.67 (6.51) (n=72)
CARAT T1 not well controlled (ACQ) CARAT T1 well controlled (ACQ)
12.77 (5.32) 21.43 (5.85)
(n=22) (n=70)
Internal consistency
In internal consistency analysis, Cronbach’s alpha for the total CARAT questionnaire was 0.82 on T1, with an alpha of 0.81 for Factor 1 (allergic rhinitis) and 0.77 for Factor 2 (asthma). Cronbach’s alpha for the total CARAT questionnaire was 0.86 on T2 and 0,83 on T3. Construct validity
Spearman correlation coefficients between CARAT scores and its separate factors and ACQ and VAS scores are listed in Table 4. All correlations proved to be statistically significant (p < 0.01). The most relevant correlations according to symptomatology categories are shown bold. These four correlations and the correlation of the total CARAT score with the ACQ score are visualized in scatter plots 1-4. Note that correlation coefficients are remarkably lower when comparing factors and questions that cover different symptoms (lower versus higher airway symptoms). Table 4. Spearman correlations construct validity
T1 ACQ VAS1 VAS2 VAS3 VAS2+3 CARAT total score -0.66 -0.69 -0.62 -0.64 -0.76
CARAT factor 1 (allergic rhinitis) -0.41 -0.47 -0.41 -0.70 -0.66
CARAT factor 2 (asthma) -0.70 -0.68 -0.64 -0.44 -0.71
Table 4. All correlations were statistically significant (p < 0.01).
12
Scatter plots 1-4. Correlations construct validity
Longitudinal validity
Spearman correlation coefficients between delta scores for the CARAT and delta scores for the ACQ and VAS scores are listed in Table 5. Delta 1 represents the four week period between T1 and T2, delta 2 represents the four week period between T2 and T3. All correlations proved to be statistically significant (p < 0.01). Longitudinal correlations for total CARAT scores are visualized in scatter plots 5-8.
13
Table 5. Spearman correlations longitudinal validity
Delta 1 (T1-T2) Delta 2 (T2-T3)
CARAT total scores
CARAT - ACQ 0.45 0.40
CARAT - VAS1 0.61 0.45
CARAT - VAS2+3 0.67 0.40
CARAT factors
Factor 1 (allergic rhinitis) - VAS3 0.55 0.36
Factor 2 (asthma) - VAS2 0.45 0.29
Factor 2 (asthma)- ACQ 0.41 0.41
Table 5. All correlations proved to be statistically significant (p < 0.01).
Scatter plots 5-8. Correlations longitudinal validity
14
Minimal clinically important difference
The minimal clinically important difference (MCID) was calculated using the GRC scores that were measured on T2, thus representing the four week period between T1 and T2. The different categories are listed in Table 6. The MCID for total CARAT scores, derived from the ‘minimal difference’ group, is 3.50 shown bold in the table. The maximum CARAT score being 30 points, the MCID is 11.6% of the total CARAT score.
Table 6. MCID
Difference category Global rating of change | GRC |
Difference CARAT score Mean (SD)
None (n=44) 0-1 3.00 (2.79)
Minimal (n=16) 2-3 3.50 (2.78)
Moderate (n=15) 4-5 7.07 (4.08)
Large (n=7) 6-7 5.57 (6.40)
Test-retest reliability Test-retest reliability was calculated for a sample of patients that were in the ‘No difference’ category based on their absolute GRC scores for T1-T2 and T2-T3. Comparing the CARAT score for these two groups there was a Spearman’s correlation coefficient of 0.74 for T1-T2 (n=44) and 0.78 for T2-T3 (n=31). Both correlations were statistically significant (p < 0.01). Discriminative properties On T1, the group with ‘well controlled’ disease (n=70) according to the ACQ had a mean CARAT score of 21.43 (SD 5.85), whereas the ‘not well controlled’ group according to the ACQ (n=22) had a mean CARAT score of 12.77 (SD 5.32). The difference in means between both groups was significant (p < 0.001). Group means are listed in Table 3.
15
Pollen counts Delta CARAT scores for the four week period prior to T1 and T2 showed correlation coefficients ranging from 0.14 to 0.32, when compared to delta pollen counts in the same period. For T2-T3, delta CARAT scores and delta pollen counts showed correlation coefficients ranging from 0.08 to 0.17. Correlation coefficients between delta CARAT scores and delta pollen counts are listed in Table 7. Different samples were used according to diagnosis and clinical reactivity. Delta CARAT scores and delta pollen counts for patients with clinical reactivity to at least one pollen species were normally distributed and therefore analyzed using Pearson correlation coefficients. All other results are Spearman correlation coefficients.
Table 7. Correlation of delta CARAT with delta pollen counts
Delta pollen counts T1-T2 Delta pollen counts T2-T3
Delta CARAT score 0.20 (p < 0.01) 0.13 (p < 0.01)
Delta CARAT Factor 1(allergic rhinitis) 0.14 (p < 0.01) 0.08 (p < 0.01)
Diagnose allergic rhinitis (n= 49) Delta CARAT score
0.23 (p < 0.01)
0.17 (p < 0.01)
Clinical reactivity to pollen (n= 41) Delta CARAT score
0.32 (p < 0.05)
0.10 (p < 0.05)
Table 7. The ‘clinical reactivity to pollen’ group was compared to delta pollen counts using
Pearson correlation coefficients. Other listed results are Spearman correlation coefficients.
16
Discussion
The CARAT is the first questionnaire to monitor the control of allergic rhinitis and
asthma simultaneously. This article describes the validation and evaluation of the
Dutch version of the CARAT. This study found that the Dutch CARAT is a valid tool
with good consistency and discriminative properties. Accordingly, the CARAT is
useful in the Netherlands to facilitate optimal control of both asthma and allergic
rhinitis simultaneously, which has been extensively proposed (1,16).
The Dutch CARAT is shown to have high internal consistency. The Cronbach’s alpha
for all of the three measurements is comparable to the alpha that was found in the
Portuguese CARAT study (6). Also, both Factor 1 (allergic rhinitis) and Factor 2
(asthma) individually show high internal consistency, suggesting both asthma and
allergic rhinitis questions are comparably consistent.
Due to the absence of a gold standard, the ACQ and VAS scores concerning lower
and higher airway symptoms were used to evaluate construct validity of the CARAT.
Correlations between both total scores and scores for different domains of the
questionnaires were calculated. All a priori expectations were met. The total CARAT
score has good construct validity when compared to the total score of the ACQ and
the VAS questions that cover both lower and higher airway symptoms. Comparing
only Factor 2 of the CARAT to the ACQ shows better correlation than the total
CARAT score. This result was expected, given the fact ACQ covers asthma
symptoms exclusively, as does CARAT Factor 2. The same effect was seen in the
correlation between CARAT Factor 2 and VAS question 2, which is exclusively about
higher airway symptoms. CARAT Factor 1 showed especially good correlation with
VAS question 3, which covers higher airway symptoms. Comparing lower and higher
airway domains vice versa showed lower correlation coefficients throughout. Results
were similar to findings for the Portuguese version of the questionnaire (6),
underlining good construct validity of the CARAT.
The CARAT study is shown to have satisfactory longitudinal validity. Correlation
coefficients between Delta scores of the CARAT and Delta scores of the ACQ and
VAS questions were reasonably high. Comparing total scores showed higher
correlation coefficients than the individual sub domains of the questionnaires. Also,
Delta 1 (T1-T2) showed better correlation throughout when compared to Delta 2 (T2-
T3). This may be explained by the fact that the amount of reminders that had to be
sent to participants was substantially bigger for T3 compared to T2 and T1, which
was probably due to the summer holiday (this is shown in Table 1). This resulted in
more individual variability in the period between T2 and T3 (Delta 2) Another reason
for Delta 2 correlating less convincingly than Delta 1 might be the smaller number of
participants in T3, resulting in less power. Overall longitudinal validity was as
expected. Correlation coefficients that were calculated between all Delta scores are
17
satisfactorily when evaluating quality of life scales, because scores can be easily
impacted by patient and external influences.
Minimal clinically important difference for the CARAT was evaluated. The ‘minimal
difference’ group shows a mean CARAT score difference of 3.50. This number
represents the difference in CARAT score for one patient over time that the patient
perceives as relevant and is therefore a clinically relevant finding. The minimal
clinically important difference will aid clinicians in interpreting the importance of
change in asthma and allergic rhinitis patients. The unexpected mean difference in
CARAT score for the ‘large difference’ group is assumed to be due to the small
sample size.
Test-retest reliability for the Dutch CARAT is shown to be high. CARAT scores on
different measuring moments were compared, only using the ‘no difference’ group
(based on GRC scores) in analysis. For these participants with no change or hardly
any change in symptoms, CARAT scores correlate very well individually.
The Dutch CARAT is shown to be able to distinguish patients based on ACQ scores.
Patients in the lower scoring category showed a mean CARAT score that is
significantly lower that the mean CARAT scores of the patients in the higher scoring
category. Therefore, discriminative properties of the Dutch CARAT are good when it
comes to distinguishing “well controlled” patients from “not well controlled” patients
based on ACQ scores (22).
The sensitivity of the CARAT questionnaire for detecting environmental influences on
asthma and allergic rhinitis was evaluated. Correlations between pollen counts during
the course of the study and CARAT scores did not show the expected consistency.
Both periods (T1-T2, T2-T3) show differing correlations, when looking at total CARAT
scores and scores of Factor 1, the allergic rhinitis subdomain. Correlation is higher
when using a sample of patients who are diagnosed with allergic rhinitis. Ultimately a
correlation coefficient of 0.32 was found for the first four week period by using a
sample of patients who were clinically allergic to at least one species of pollen. The
correlation coefficient for the second period was markedly lower. This is expected to
be due to the same factors that were discussed for longitudinal validity. The
correlation coefficient for the first period is not persuasively high, though it certainly
suggests a relation between pollen counts and changes in CARAT score. High
correlations cannot be expected given the fact that pollen counts were measured in
Leiden while participants were located elsewhere. Also, all allergenic pollen species
were treated equally in statistical analysis, not accounting the different degrees of
allergenicity. For more concrete results, further analysis of the measured pollen
counts is required to further explore relations between pollen counts and changes in
CARAT scores. An important step for this would be to weigh pollen counts differently,
in order to give more allergenic pollen species more impact in all calculations. Also,
mean pollen counts for a shorter period (several days) before each measurement
could be used for analyses, based on the hypothesis that pollen counts and therefore
18
symptoms of the last few days have more impact on the way participants fill out the
questionnaire. It could also be useful to let participants fill out questionnaires on the
same date, so intervals between measurements are actual periods rather than virtual
periods. This could result in larger and more consistent differences in pollen counts if
the first measurement is made before, and the second during the peak of allergic
rhinitis season.
Patient characteristics show a female predominance in the sample, with lower mean
CARAT scores than male participants. These results underline findings in the
Portuguese CARAT study. Results of skin prick tests, specific IgE and lung function
were not used for statistical analysis. This was due to the low rates of available data
concerning additional clinical research (especially in primary care patients) and the
different ways these results were presented in each center. Also, medication use was
not included in statistical analyses because of inconsistent patient data.
The sample size for each measurement (T1, T2, T3) is sufficient, given the fact that a
number of 50-100 patients is recommended for validation studies. For each
measuring moment the number of patients reduces with 10, resulting in slightly
reducing power in the course of the study. For this reason, the main focus in the
interpretation of the results lies with T1 and Delta 1 (T1-T2). It is remarkable how the
percentage of sent reminders increases during the course of the study. This is
expected to be due to the summer holiday period surrounding T3.
A limitation of the study may be the incomplete descriptive baseline characteristics
concerning lung function, specific IgE and skin prick tests, especially for primary care
patients. However, these are not standard clinical investigations in primary care. This
particular data would have been included if it were available. This particular data
could be included in further research if these clinical investigations were performed
during the course of the study
Conclusion
The Dutch version of the CARAT has high internal consistency and good construct
validity and is therefore comparable to findings according to the Portuguese CARAT.
Additionally, the Dutch CARAT proves to be longitudinally valid, with good test-retest
reliability. The determination of MCID is highly beneficial for interpreting CARAT
scores. Further research is required to find consistent correlations between pollen
counts and CARAT scores. However, the CARAT is a valid and reliable tool for use in
clinical research and practice to monitor asthma and allergic rhinitis symptoms
simultaneously.
19
References
(1) Bousquet J, Khaltaev N, Cruz AA, Denburg J, Fokkens W, Togias A, et al. ARIA update 2008: allergic rhinitis and its effect on asthma. Allergologie 2009 AUG;32(8):306-319.
(2) Bresciani M, Paradis L, Des Roches A, Vernhet H, Vachier I, Godard P, et al. Rhinosinusitis in severe asthma. J Allergy Clin Immunol 2001 JAN;107(1):73-80.
(3) Sole D, Camelo-Nunes I, Wandalsen G, Melo K, Naspitz C. Is rhinitis alone or associated with atopic eczema a risk factor for severe asthma in children? Pediatr Allergy Immunol 2005 MAR;16(2):121-125.
(4) Bousquet J, Gaugris S, Kocevar V, Zhang Q, Yin D, Polos P, et al. Increased risk of asthma attacks and emergency visits among asthma patients with allergic rhinitis: a subgroup analysis of the improving asthma control trial. Clin Exp Allergy 2005 JUN;35(6):723-727.
(5) Terreehorst I, Oosting A, Tempels-Pavlica Z, de Monchy J, Bruijnzeel-Koomen C, Hak E, et al. Prevalence and severity of allergic rhinitis in house dust mite-allergic patients with bronchial asthma or atopic dermatitis. Clin Exp Allergy 2002 AUG;32(8):1160-1165.
(6) Fonseca JA, Nogueira-Silva L, Morais-Almeida M, Azevedo L, Sa-Sousa A, Branco-Ferreira M, et al. Validation of a questionnaire (CARAT10) to assess rhinitis and asthma in patients with asthma. Allergy 2010 AUG;65(8):1042-1048.
(7) Bousquet J, Neukirch F, Bousquet P, Gehano P, Klossek J, Le Gal M, et al. Severity and impairment of allergic rhinitis in patients consulting in primary care. J Allergy Clin Immunol 2006 JAN;117(1):158-162.
(8) Walker S, Khan-Wasti S, Fletcher M, Cullinan P, Harris J, Sheikh A. Seasonal allergic rhinitis is associated with a detrimental effect on examination performance in United Kingdom teenagers: Case-control study. J Allergy Clin Immunol 2007 AUG;120(2):381-387.
(9) Bousquet J, Bodez T, Gehano P, Klossek JM, Liard F, Neukirch F, et al. Implementation of Guidelines for Allergic Rhinitis in Specialist Practices Randomized Pragmatic Controlled Trial. Int Arch Allergy Immunol 2009;150(1):75-82.
(10) Church MK, Maurer M, Simons FER, Bindslev-Jensen C, van Cauwenberge P, Bousquet J, et al. Risk of first-generation H-1-antihistamines: a GA2LEN position paper. Allergy 2010 APR;65(4):459-466.
(11) Baiardini I, Pasquali M, Giardini A, Specchia C, Passalacqua G, Venturi S, et al. Rhinasthma: a new specific QoL questionnaire for patients with rhinitis and asthma. Allergy 2003 APR;58(4):289-294.
20
(12) Kilpelainen M, Terho E, Helenius H, Koskenvuo M. Validation of a new questionnaire on asthma, allergic rhinitis, and conjunctivitis in young adults. Allergy 2001 MAY;56(5):377-384.
(13) Fischer PE, Grabbe Y, Nolting H-. Development and validation of a screening questionnaire for allergy airway diseases (ASF Screening Questionnaire). Allergologie 2006 OCT;29(10):393-402.
(14) Wasserfallen J, Gold K, Schulman K, Baraniuk J. Development and validation of a rhinoconjunctivitis and asthma symptom score for use as an outcome measure in clinical trials. J Allergy Clin Immunol 1997 JUL;100(1):16-22.
(15) Bousquet J, Bousquet P, Godard P, Daures J. The public health implications of asthma. Bull World Health Organ 2005 JUL;83(7):548-554.
(16) Braunstahl G, Fokkens W. Nasal involvement in allergic asthma. Allergy 2003 DEC;58(12):1235-1243.
(17) Bateman ED, Hurd SS, Barnes PJ, Bousquet J, Drazen JM, FitzGerald M, et al. Global strategy for asthma management and prevention: GINA executive summary. Eur Resp J 2008 JAN;31(1):143-178.
(18) Juniper E, O'Byrne P, Guyatt G, Ferrie P, King D. Development and validation of a questionnaire to measure asthma control. Eur Resp J 1999 OCT;14(4):902-907.
(19) Nathan R, Sorkness C, Kosinski M, Schatz M, Li J, Marcus P, et al. Development of the asthma control test: A survey for assessing asthma control. J Allergy Clin Immunol 2004 JAN;113(1):59-65.
(20) Vollmer W, Markson L, O'Connor E, Sanocki L, Fitterman L, Berger M, et al. Association of asthma control with health care utilization and quality of life. Am J Respir Crit Care Med 1999 NOV;160(5):1647-1652.
(21) Nogueira-Silva L, Martins SV, Cruz-Correia R, Azevedo LF, Morais-Almeida M, Bugalho-Almeida A, et al. Control of allergic rhinitis and asthma test - a formal approach to the development of a measuring tool. Respir Res 2009 JUN 17;10:52.
(22) Juniper E, Bousquet J, Abetz L, Bateman E, GOAL Comm. Identifying 'well-controlled' and 'not well-controlled' asthma using the Asthma Control Questionnaire. Respir Med 2006 APR;100(4):616-621.
(23) Bousquet P, Combescure C, Klossek J, Daures J, Bousquet J. Change in visual analog scale score in a pragmatic randomized cluster trial of allergic rhinitis. J Allergy Clin Immunol 2009 JUN;123(6):1349-1354.
(24) Jaeschke R, Singer J, Guyatt G. Measurement of Health-Status - Ascertaining the Minimal Clinically Important Difference. Control Clin Trials 1989 DEC;10(4):407-415.
21
Appendix
22
Allergie en Astma Vragenlijst voor volwassenen
De ingevulde vragenlijst graag terugsturen in de
bijgeleverde envelop. Een postzegel is niet nodig!
23
1. Hoe vaak hebt u, vanwege uw allergische neusklachten en/of astma, de laatste 4 weken
gemiddeld last gehad van:
Nooit 1 of 2
dagen per
week
Meer dan 2 dagen
per week
(Bijna) iedere
dag
1. Verstopte neus?
2. Niezen?
3. Jeuk aan de neus?
4. Loopneus?
5. Benauwdheid / Kortademigheid?
6. Piepende ademhaling?
7. Beklemd gevoel op de borst bij lichamelijke inspanning?
8. Vermoeidheid en/of beperkingen bij het uitvoeren van dagelijkse bezigheden?
9. ’s-Nachts wakker worden?
Hoe vaak moest u de laatste 4 weken, vanwege allergische neusklachten en/of astma:
Ik gebruik geen
medicijnen
Nooit Minder dan 7
dagen
7 dagen of
meer
10. Uw medicijnen extra
gebruiken?
CARAT
24
2. Zet een cirkeltje om het getal dat het beste beschrijft hoe u zich de afgelopen week heeft
gevoeld (slechts één antwoord per vraag). LET OP! Bij deze vragen gaat het om de afgelopen week
(en niet de afgelopen 4 weken).
1. Hoe vaak bent u per nacht gemiddeld wakker
geworden door uw astma in de afgelopen
week?
0. Nooit 1. Bijna nooit 2. Een paar keer 3. Verscheidene keren 4. Vaak 5. Heel vaak 6. Kon niet slapen vanwege astma
2. Hoe ernstig waren uw astma klachten bij het 's
morgens wakker worden gemiddeld in de
afgelopen week?
0. Geen klachten 1. heel lichte klachten 2. lichten klachten 3. matige klachten 4. vrij ernstige klachten 5. ernstige klachten 6. heel ernstige klachten
3. In welke mate werd u over het algemeen in de
afgelopen week door uw astma beperkt bij uw
activiteiten?
0. helemaal niet 1. nauwelijks 2. een beetje 3. tamelijk 4. erg 5. heel erg 6. volledig
4. In welke mate heeft u zich over het algemeen
kortademig gevoeld in de afgelopen week ten
gevolge van uw astma?
0. helemaal niet 1. nauwelijks 2. een beetje 3. middelmatig 4. vrij ernstig 5. ernstig 6. heel ernstig
5. Hoe vaak had u in de afgelopen week over het
algemeen een piepende ademhaling?
0. nooit 1. zelden 2. af en toe 3. geregeld 4. vaak 5. meestal 6. altijd
6. Hoe veel pufs/inhalaties van een kortwerkend
luchtwegverwijdend middel (bijv. ventolin of
bricanyl) heeft u op de meeste dagen
genomen in de afgelopen week?
0. geen 1. 1-2 pufs/inhalaties 2. 3-4 pufs/inhalaties 3. 5-8 pufs/inhalaties 4. 9-12 pufs/inhalaties 5. 13-16 pufs/inhalaties 6. meer dan 16 pufs/inhalaties
ACQ
Juniper EF, O'Byrne PM, Guyatt GH, Ferrie PJ, King DR. Development and validation of a questionnaire to measure asthma
control. Eur Respir J. 1999;14(4):902–7.
Fonseca JA, Nogueira-Silva L, Morais-Almeida M, Azevedo L, Sa-Sousa A, Branco-Ferreira M, Fernandes L, Bousquet J. Validation
of a questionnaire (CARAT10) to assess rhinitis and asthma in patients with asthma. Allergy. 2010 Aug;65(8):1042-8.
25
3. Hieronder ziet u een lijn die loopt van ‘geen klachten’ tot ‘extreem veel klachten’. Wilt u per
vraag op de lijn een kruis zetten op de plek die het beste weergeeft hoeveel klachten u de
afgelopen 4 weken heeft gehad? Bijvoorbeeld;
Geen klachten I------------------------------------------X----------------------------------I Extreem veel
1. Hoeveel klachten van uw luchtwegen in het algemeen had u de afgelopen 4 weken?
Geen klachten I-------------------------------------------------------------------------------I Extreem veel
2. Hoeveel klachten van uw longen (astma) had u de afgelopen 4 weken?
Geen klachten I-------------------------------------------------------------------------------I Extreem veel
3. Hoeveel klachten van uw neus (hooikoorts) had u de afgelopen 4 weken?
Geen klachten I-------------------------------------------------------------------------------I Extreem veel
0 1 2 3 4 5 6 7 8 9 10 klachten
0 1 2 3 4 5 6 7 8 9 10 klachten
0 1 2 3 4 5 6 7 8 9 10 klachten
0 1 2 3 4 5 6 7 8 9 10 klachten
VAS
26
4. Is er enige verandering in uw allergische neusklachten en/of astma sinds de vorige keer dat u
deze vragenlijsten heeft ingevuld (ongeveer een maand geleden)? Zet een cirkeltje om het getal dat
het beste weergeeft hoeveel slechter of beter het nu gaat. Slechts één antwoord mogelijk.
De allergische neusklachten en/of astma is:
-7. extreem veel slechter
-6. erg veel slechter
-5. veel slechter
-4. redelijk veel slechter
-3. enigszins slechter
-2. een beetje slechter
-1. ongeveer hetzelfde, nauwelijks slechter
0. hetzelfde
1. ongeveer hetzelfde, nauwelijks beter
2. een beetje beter
3. enigszins beter
4. redelijk veel beter
5. veel beter
6. erg veel beter
7. extreem veel beter
GRC
27
5. Heeft u de afgelopen 4 weken medicijnen gebruikt voor uw neusklachten en/of astma?
Nee Ja Vul hieronder naam, soort en gebruik in
Naam Medicijn Toedieningsvorm * Sterkte Hoe vaak gebruikt u deze?
T Z P I
T Z P I
T Z P I
T Z P I
T Z P I
* Tablet = T, Zalf of crème = Z, Pufje = P, Injectie = I
Tenslotte volgen hier nog een aantal algemene vragen. Graag volledig invullen. Deze gegevens zijn
van belang voor het onderzoek en worden gebruikt om de vragenlijst te coderen.
6. Datum van invullen …. - …. - …….. (dd-mm-jjjj)
7. Geboorte datum …. - …. - …….. (dd-mm-jjjj)
8. Geslacht Man Vrouw
Algemeen
Dit is het einde van de vragenlijst.
Zou u de vragenlijst zo spoedig mogelijk willen terugsturen in de bijgevoegde enveloppe?
Een postzegel is niet nodig!
Hartelijk dank voor het invullen!