development and standardization of tinnitus handicap inventory in...
TRANSCRIPT
-
Asia Pacific Journal of Research
ISSN (Print): 2320-5504
ISSN (Online): 2347-4793
www.apjor.com Vol: I. Issue LII, June 2017
10
DEVELOPMENT AND STANDARDIZATION OF TINNITUS HANDICAP
INVENTORY IN BANGLA
Pinaki Dutta1, Indranil Chatterjee
1, Sujoy Kumar Makar
1, Craig W. Newman
2, Arpita Chatterjee
1
1Department of Audiology
Ali Yavar Jung National Institute of Speech and Hearing Disabilities, Eastern Regional Centre, Kolkata-90
2Cleveland Clinic, USA
ABSTRACT
The aim of the study was to develop and standardize a Bangla version of the Tinnitus Handicap Inventory (THI) and to determine
the validity, internal consistency reliability and test-retest reliability of the Bangla version of the Tinnitus Handicap Inventory
(THI-Bangla) for clinical and research purposes in Bengali population. The THI-Bangla was administered to thirty patients
suffering from tinnitus, aged between 30 to 45 years. The clients were selected from the audiological department of AYJNISHD,
ERC Kolkata. The client having tinnitus without any medical problem was included in the group. The correlation between THI-
Bangla scores and matched loudness of tinnitus showed acceptable correlation (r=0.614). In case of correlation of THI- Bangla
and matched pitch of the participants the result showed r=0.621 which indicates an acceptable correlation. Inter item correlation
was calculated using the Cronbachs alpha test indicated the inter item correlation of total THI was 0.845 the inter-item
correlation of functional, emotional and catastrophic sub scale were, 0.850, 0.813 and 0.637, respectively. The Pearson product-
moment correlation was done among the sub scales. The association between functional and emotional sub-scale was high
(r=0.951). The association between the emotional and catastrophic sub-scale were good (r= 0.889). Internal association was
observed between functional and catastrophic sub-scale were also high (r= 0.987) Kendalls tau_b test was administered to
identify the test-retest reliability. The correlation coefficient was 1.000, which indicates, correlation is significant at 0.01 levels.
This study demonstrated that the Bangla version of THI (THI- Bangla) is a valid and reproducible tool to assess the psychosocial
complaints of tinnitus sufferers.
Key Words : Tinnitus , Handicap ,
INTRODUCTION
Tinnitus is a subjective symptom making its assessment a challenging task for the audiologists. A number of measures have been
developed to assess tinnitus-related distress, coping strategies, and cognitive reactions to tinnitus (Hallam, Jakes & Hinchcliffe,
1988; Wilson & Henry, 1998). In this connection self-reporting tinnitus questionnaires are one of the most commonly used tools
for assessing the way patients experience tinnitus (Wilson & Henry, 2000). Currently, there exist numerous questionnaires that are
available to assess the quality of life of tinnitus in such manner. One of the most widely used questionnaires is Tinnitus Handicap
Inventory (THI) (Newman, Jacobson & Spitzer, 1996). The degree of distress, annoyance, emotional discomfort, sleep problems
and interference with day-to-day activities are factors that differentiate people who simply experience tinnitus from those who
need help and clinical attention (i.e., have clinically significant tinnitus) (Jastreboff & Hazell, 2004).
Most of the instruments that evaluate the Quality of Life were developed in English and were intended to be used in English
speaking countries. There is a need to develop instruments to be used in countries where English is not spoken because; such
questionnaires when used with the native speakers of Bangla will not yield exact results if the psychometric adequacy of the tool
is not investigated. In other words when such questionnaires are used with the native speakers of Bangla, they may not provide
reliable and valid estimation of the psychosocial consequences of tinnitus. Therefore, a new instrument may be developed or an
-
Asia Pacific Journal of Research
ISSN (Print): 2320-5504
ISSN (Online): 2347-4793
www.apjor.com Vol: I. Issue LII, June 2017
11
existing one may be translated into another language (Guillemin, Bombardier & Beaton, 1993), by which, tinnitus related activity
limitation and participate restriction of an individual speaking Bangla could be measured appropriately.
In this study the major challenge was to assess the handicap impact of the Bangla speaking person having tinnitus through a
quality of life questionnaire. The THI was selected for adaptation because it is a brief and easy-to-administer questionnaire that is
suitable for use in busy clinical settings (Newman, Jacobson & Spitzer, 1996). Harris and Mc Ghee-Nelson, 1992 had stated that,
bilingualism and multilingualism are the norm rather than the exception in todays world. India, not being an exception is also a
multilingual and multicultural country, where only 5- 10% of population can understand English (Bharati & Kulkarni, 2005).
However there is no known self-rating questionnaire in Bangla language available to assess the handicapping impact due to
tinnitus. Accordingly, there a need to develop and standardize such a well-designed questionnaire translated into an Indian
language (Bangla), which can assess quickly and accurately the trouble of the person experiencing tinnitus.
The overall aim of the study was to develop the Bangla Tinnitus Handicap Inventory (THI-Bangla). More specifically, the study
was undertaken to standardize the THI- Bangla by determining its validity, internal consistency reliability and test-retest
reliability.
METHOD
Participant Selection: Thirty native Bangla speaking individuals having tinnitus was taken for the study. The individuals had a
history of permanent and spontaneous, unilateral or bilateral subjective tinnitus experiencing from at least last six months. These
subjects also had mild to moderately severe sensorineural hearing loss. The age range was from 30 to 45 years (mean age:
40.43years, standard deviation of age: 13.77). This study group included participants who were able to speak and read both
Bangla and English adequately and the minimum educational qualification was matriculation. All the participants hailed from
similar socio-economic background.
Inclusion criteria: This was determined through case history and audiological evaluation. Information was obtained on- whether
the clients had history of diabetes, presence of any type of noise exposure/ acoustic trauma or use of ototoxic drugs. The presence
of hearing loss and the type of hearing loss was assessed by pure tone audiometry. To rule out the middle ear pathology of the
client having tinnitus, tympanometry was done. Otoacoustic emission (OAE) and auditory brainstem response (ABR) were used to
rule out the auditory neuropathy. Loudness discomfort level (LDL) testing was also done to identify the discomfort level of the
client. The client having tinnitus without middle ear pathology, cardiovascular diseases, neurologic diseases, diabetes, Mennieres
disease, noise exposure or acoustic trauma, hyperacusis was included in the group.
To ascertain whether the client really had tinnitus, Tinnitus characteristics were recorded using Tinnitus Psychoacoustic
Assessment (Vernon & Meikle, 1981) using following steps: identification of the tinnitus ear, understanding of the terms
loudness and pitch, pitch matching, loudness matching.
Instrumentation: The clinical evaluation was conducted using a diagnostic audiometer (MAICO MA53) with TDH 39 earphones,
and calibrated (according to manufacturers standards), immittance audiometer (GSI-39 AUTO TYMP), otoacoustic emission
system (MAICO ERO SCAN), and auditory brainstem response instrumentation (RMS Medulla AD).
Procedure to Develop Bangla Tinnitus Handicap Inventory Material:
Phase 1: Development of test material by linguistic validation
The linguistic validation (Guillemin, Bombardier & Beaton, 1993) of the original version of the Tinnitus Handicap Inventory
(THI-US) (Newman, Jacobson & Spitzer, 1996) into the Bangla version of Tinnitus Handicap Inventory (THI-Bangla) and its
psychometric specification was completed using the following steps:
(a) Translation: In the first step THI questionnaire (appendix 1) was translated by ten native speakers of Bangla, with high level of
proficiency in English and Bangla. These ten native speakers translated the original instrument into Bangla language individually.
(b) Back Translation: As the second step of linguistic validation, the services of another set of ten translators with high level of
proficiency in Bangla and English languages were utilized. Back translation of the pooled Bangla version to English was done to
measure the homogeneity of the original version of THI.
Phase 2: Pilot testing
The pooled back translation was compared with the original version of THI by three linguists and three audiologists. Finally, the
initial Bangla version of the THI instrument (appendix 2) was formulated with proper and satisfactory translation and after item
analysis.
To establish construct validity, the total score and each subscale score (i.e. Functional, Emotional and catastrophic) was correlated
with matched mean pitch and mean loudness of the subjects. This questionnaire was used as an initial instrument for gathering
data from tinnitus subjects.
Phase 3: Participant selection
The participants were selected from the audiological department of AYJNIHH, ERC and different hearing clinics of Kolkata,
India. Participants fulfilling the inclusion criteria for the study were selected randomly.
Phase 4: Administering the developed test on participants with tinnitus and obtaining the test score
-
Asia Pacific Journal of Research
ISSN (Print): 2320-5504
ISSN (Online): 2347-4793
www.apjor.com Vol: I. Issue LII, June 2017
12
The validated THI- Bangla material was given to each participant who was ask to respond to each item on the questionnaire by
indicating a choice of "Yes", "No" and "Sometimes". The score of "Yes" is 4, "No" is 0 and "Sometimes" is 2. Finally, the
total THI score was measure by (number of Yes responses x 4) + (number of Sometimes responses x 2).
Phase 5: Evaluating the reliability of the test
(a) Internal consistency: To measure the homogeneity of three sub scale items, in terms of within subscales correlations.
(b) Test-retest reliability: To evaluate its test-retest reliability, after one week from completing the first THI-Bangla, the
questionnaire was re-administered to the same patients. The retest was done without informing the patients of the score of the
previous test. There were no intervening treatment between test and retest conditions, e.g., fitting of hearing aid or noise generator
devices, drug trials or any psychological management.
The data obtained from the participants entered into a Microsoft office excel 2007 database, and was subsequently subjected to
statistical analysis conducted using Statistical Package for Social Sciences (SPSS), version 16.0 include:- Cronbachs alpha ,
b)Pearsons product moment correlation , c) Kendalls tau b correlation
RESULTS
Validity: The six judges consisted of three linguists and three audiologists. A three point rating scale (1= Not valid, 2= Valid, 3=
Most valid) was used for validating the THI- Bangla. The pictorial representation of the judges response for 25 items of Tinnitus
Handicap Inventory to validate the assessment tool is given in appendix 3.
After validation of the twenty five items of THI- Bangla questionnaire (appendix 2), internal consistency and reliability of this
questionnaire were examined. For standardizing THI- Bangla detailed results of construct validity, internal consistency and
reliability measures are documented below.
Construct validity: as shown in Table 1, the result of Pearsons product moment correlation among the THI- Bangla,
matched loudness and pitch of tinnitus subject were high.
Table 1: Correlation of Tinnitus Handicap Inventory Bangla with matched pitch and loudness of tinnitus sufferers
THI- Bangla score Loudness matching Pitch matching
THI- Bangla score r: 1 - -
Loudness matching r: 0.614 r: 1 -
Pitch matching r: 0.621 r: 0.565 r: 1
The correlation between THI- Bangla scores and matched loudness of tinnitus shows acceptable correlation (r: 0.614). In case of
correlation of THI- Bangla and matched pitch of the participants the value obtained was r: 0.621 which again indicates an
acceptable correlation.
Reliability: To evaluate the inter-item correlation of the full THI - Bangla, inter item correlation was done among the twenty-five
questionnaire of THI- Bangla using Cronbach's Alpha test (Table 2). The result of the Cronbach's Alpha is 0.845, which indicates
the questionnaires have good inter-item correlation as may be seen in table 2.
Table 2: Inter-item correlation of Tinnitus Handicap Inventory Bangla (THI - Bangla)
Cronbach's
Alpha
Cronbach's Alpha Based on
Standardized Items
Mean Variance Std.
Deviation
No. of Items
0.845 0.844 71.8333 201.802 14.20571 25
Inter-item correlations of three sub-scales of Tinnitus Handicap Inventory were done using Cronbach's alpha. The results of inter-
item correlation of each sub-scale are displayed in table 3, 4 and 5.
Table 3: Inter-item correlation of functional sub-scale of Tinnitus Handicap Inventory Bangla (THI - Bangla)
Cronbach's
Alpha
Cronbach's Alpha Based on
Standardized Items
Mean Variance Std.
Deviation
No. of Items
0.850 0.853 34.0667 85.148 9.22757 12
The functional scale of THI Bangla, includes twelve questionnaires. Inter-item correlation among the twelve questionnaires was
calculated using Cronbachs alpha to identify the internal consistency of functional sub-scale. Cronbachs alpha value of 0.850
was obtained which indicates good inter-item correlation among the twelve questionnaires of the functional sub-scale.
-
Asia Pacific Journal of Research
ISSN (Print): 2320-5504
ISSN (Online): 2347-4793
www.apjor.com Vol: I. Issue LII, June 2017
13
Table 4: Inter-item correlation of emotional sub-scale of Tinnitus Handicap Inventory Bangla (THI - Bangla)
Cronbach's
Alpha
Cronbach's Alpha Based on
Standardized Items
Mean Variance Std.
Deviation
No. of Items
0.813 0.815 24.0333 41.558 6.44656 8
The emotional sub-scale consists of eight questionnaires. To identify the inter-item correlation of emotional sub-scale, Cronbachs
alpha was done. The result of internal consistency of eight questionnaire shows Cronbachs alpha value is 0.813 indicating good
internal consistency of emotional sub-scale.
Table 5: Inter-item correlation of catastrophic sub-scale of Tinnitus Handicap Inventory Bangla (THI - Bangla)
Cronbach's
Alpha
Cronbach's Alpha Based on
Standardized Items
Mean Variance Std.
Deviation
No. of Items
0.637 0.636 13.7333 14.707 3.83502 5
Five questionnaires are included in catastrophic sub scale. The internal consistency was checked among the questionnaire of
catastrophic sub-scale using Cronbachs alpha. The Cronbachs alpha value computed was 0.637 which indicates acceptable inter-
item correlation of catastrophic sub-scale. In this study the Cronbachs alpha score for the catastrophic sub-scale is less. It can be
happened due to less item (Questionnaire) is included in the catastrophic sub-scale.
The Pearson product-moment correlation between sub-scales of Tinnitus Handicap Inventory Bangla (THI- Bangla) indicates
excellent internal consistency reliability between the sub-scales. The internal consistency between functional and emotional sub-
scale shows excellent correlation (r= 0.951). The p value was 0.201, which also supports that there is a correlation between the
functional and the emotional sub-scale. The internal consistency reliability of emotional and catastrophic sub-scale shows good
correlation (r= 0.889). The p value was 0.303 an indication of more or less good correlation. Internal consistency reliability
between functional and catastrophic sub-scale shows an excellent correlation (r= 0.987) between the two sub-scales. The p value
also supports the excellent correlation between this two sub-scale (p= 0.102).
Table 6: Inter-subscale correlation of Tinnitus Handicap Inventory Bangla (THI - Bangla)
Functional Emotional Catastrophic
Functional r : 1 _ _
Emotional r : 0.951 r : 1 _
Catastrophic r : 0.987 r : 0.889 r : 1
For better precision in terms of reliable estimate test retest reliability was further done to re-confirm internal consistency measures
which have been already mentioned. The result of the test-retest reliability is given below.
Table 7: Test retest responses for the sub-scales of Tinnitus Handicap Inventory- Bangla (THI- Bangla)
Responses on initial test Responses on retest after 7 days
Yes Occasionally No Yes Occasionally No
Functional 175 159 26 172 160 28
Emotional 136 89 15 136 88 16
Catastrophic 69 70 11 65 74 11
To check the test retest reliability, Kendall's tau_b test was administered. The result shows the correlation coefficient is 1.000,
which indicates, correlation is significant at 0.01 levels. That indicates there is a high correlation between the responses of clients
having tinnitus on the initial and the subsequent administration of the test.
DISCUSSION
The phenomenon of tinnitus extends across several psychosocial domains that affect the everyday function of patients with
tinnitus. The THI focuses on three domains of the clients having tinnitus, namely functional, emotional and catastrophic domain.
The THI is used worldwide and has been adapted and developed in various languages. The internal consistency of the total scores
as well as scores of functional sub-scale, scores of emotional sub-scale and the scores of the catastrophic sub-scale are represented
in Table 8. From the table the difference between the inter-item correlations of various languages may be appreciated as reported
in various studies.
-
Asia Pacific Journal of Research
ISSN (Print): 2320-5504
ISSN (Online): 2347-4793
www.apjor.com Vol: I. Issue LII, June 2017
14
Table 8: Cronbachs alpha scores of Tinnitus Handicap Inventory for various languages
Total Functional Emotional Catastrophic
THI- US = 0.930 = 0.860 = 0.870 =0.680
THI- Danish = 0.930 = 0.870 = 0.880 = 0.740
THI- Brazilian = 0.929 - - -
THI- Turkish = 0.886 = 0.780 = 0.750 = 0.800
THI- Italian = 0.910 = 0.850 = 0.860 = 0.630
THI- Chinese (Cantonese) = 0.940 = 0.890 = 0.890 = 0.640
THI- Persian = 0.943 - - -
THI- Chinese (Mandarin) = 0.930 = 0.850 = 0.870 = 0.780
THI- Filipino = 0.920 = 0.860 = 0.820 = 0.630
THI- Thai = 0.902 = 0.804 = 0.831 = 0.661
THI- Tamil =0.960 = 0.970 = 0.960 = 0.930
THI- Bangla = 0.845 = 0.850 = 0.813 = 0.637
From table 8, the difference between Cronbachs alpha scores of Tinnitus Handicap Inventory Bangla to main version of
Tinnitus Handicap Inventory (US) and Tinnitus Handicap Inventory developed in other languages is apparent. This supports the
notion that, the inter item correlation between the questionnaire are quite good in Tinnitus Handicap Inventory Bangla. The table
indicates that, there are very less difference between the internal consistency for Tinnitus Handicap Inventory Bangla and the
Tinnitus Handicap Inventory developed in other languages.
Table 9: The inter sub-scale correlation of Tinnitus Handicap Inventory in different languages
Functional-Emotional Emotional-Catastrophic Functional- Catastrophic
THI- US r= 0.750 r= 0.780 r= 0.650
THI- Danish r= 0.820 r= 0.740 r= 0.670
THI- Turkish r= 0.740 r= 0.590 r= 0.440
THI- Italian r= 0.500 r= 0.520 r= 0.610
THI-Chinese (Cantonese) r= 0.810 r= 0.720 r= 0.660
THI- Tamil r= 0.411 r= 0.523 r= 0.313
THI- Bangla r= 0.951 r= 0.889 r= 0.987
Pearson correlation was done to identify the correlation between the sub-scales. From table 9, it is clear that the correlation
between functional and emotional sub-scale is 0.951, which indicates an excellent correlation between functional and emotional
sub scale. Emotional and catastrophic sub-scale showed good correlation (r= 0.889). The internal consistency of functional and
catastrophic sub scale indicates an excellent correlation (r= 0.987). This type of good correlation indicated the correlation among
three sub-scales of Tinnitus Handicap Inventory Bangla is much more reliable than the Tinnitus Handicap Inventory developed
in other languages.
Kendall's tau_b test was administered to determine the test retest reliability. Correlation coefficient obtained was 1.000, which
was significant at 0.01 levels. There was thus high correlation between the responses of clients on the test administered on two
occasions separated by a week. Kam et al. (2009) identified the test retest reliability of the Chinese (Cantonese) version of
Tinnitus Handicap Inventory which indicated a good correlation (r= 0.88) between test and retest score. The test-retest score of
Chinese (Mandarin) version of Tinnitus Handicap Inventory (Meng et al., 2012) indicated an excellent correlation (r= 0.98)
between test and retest score. Mahmoudian et al. (2011) found an excellent test-retest reliability (r= 0.96) between test and retest
score of Persian version of Tinnitus Handicap Inventory (Mahmoudian et al., 2011).
CONCLUSION
This study demonstrated that the Bangla version of THI (THI- Bangla) is a valid and reproducible tool to assess the psychological
complaints of tinnitus sufferers. Also, THI- Bangla could be administered as a standard measuring instrument for patients who are
native speakers of Bangla.
-
Asia Pacific Journal of Research
ISSN (Print): 2320-5504
ISSN (Online): 2347-4793
www.apjor.com Vol: I. Issue LII, June 2017
15
REFERENCE
1. Bharati, L., & Kulkarni, A. (2005). English from hindi viewpoint: A paaninian perspective. Paper presented at platinum jubilee conference of linguistic society of India, University of Hyderabad: India.
2. Guillemin, F., Bombardier, C., & Beaton, D. (1993). Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. Journal of Clinical Epidemiology, 46(12), 1417-1432.
3. Hallam, R. S., Jakes, S. C., & Hinchcliffe R. (1988). Cognitive variables in tinnitus annoyance. British Journal of Clinical Psychology, 27(3), 213-222.
4. Harris, R. J., & McGhee Nelson, E. M. (1992). Bilingualism: Not the exception any more. In R. J. Harris & E. M. McGhee Nelson (Eds.), Cognitive processing in bilinguals, (pp. 3-14). North Holland: Elsevier.
5. Jastreboff, P. J., & Hazell, J. W. P. (2004). Tinnitus Retraining Therapy: Implementing the Neurophysioloigical Model (1st ed.). New York: Cambridge University Press.
6. Kam, A. C. S., Cheung, A. P. P., Chan, P. Y. B., Leung, E. K. S., Wong, T. K. C., Hasselt C.A., & Tong, M. C. F. (2009). Psychometric properties of the Chinese (Cantonese) Tinnitus Handicap Inventory. Clinical Otolaryngology,
34(4), 309315.
7. Mahmoudian, S., Shahmiri, E., Rouzbahani, M., Jafari. Z., Keyhani, M., Rahimi, F., Mahmoudian, G., Akbarvand, L., Barzegar, G., & Farhadi, M., (2011). Persian language version of the "Tinnitus Handicap Inventory": translation,
standardization, validity and reliability. International Tinnitus Journal, 16(2), 93-103.
8. Meng, Z., Zheng, Y., Liu, S., Wang, K., Kong, X., Tao, Y., Xu, K., & Liu, G., (2012). Reliability and Validity of the Chinese (Mandarin) Tinnitus Handicap Inventory. Clinical Exp Otorhinolaryngology, 5(1), 10-16.
9. Newman, C. W., Jacobson, G. P., & Spitzer, J. B. (1996). Development of the tinnitus handicap inventory. Arch Otolaryngology Head Neck Surgery, 122 (2), 143148.
10. Tobias, C. A., Llanes, E., Gonzalo, D. V., & Chiong, C. (2012). Validity of a Filipino translation of the Tinnitus Handicap Inventory. International Tinnitus Journal, Vol. 17(1), 9-64.
-
Asia Pacific Journal of Research
ISSN (Print): 2320-5504
ISSN (Online): 2347-4793
www.apjor.com Vol: I. Issue LII, June 2017
16
APPENDIX- I
TINNITUS HANDICAP INVENTORY US
Name: Age:
Case no: Sex:
INSTRUCTIONS: The purpose of this questionnaire is to identify difficulties that you may be experiencing because of your
tinnitus. Please answer every question. Please do not skip any questions.
No. Questions Yes Sometimes No
1. Because of your tinnitus, is it difficult for you to concentrate?
2. Does the loudness of your tinnitus make it difficult for you to hear people?
3. Does your tinnitus make you angry?
4. Does your tinnitus make you feel confused?
5. Because of your tinnitus, do you feel desperate?
6. Do you complain a great deal about your tinnitus?
7. Because of your tinnitus, do you have trouble falling to sleep at night?
8. Do you feel as though you cannot escape your tinnitus?
9. Does your tinnitus interfere with your ability to enjoy your social activities (such as going out to dinner, to the movies)?
10. Because of your tinnitus, do you feel frustrated?
11. Because of your tinnitus, do you feel that you have a terrible disease?
12. Does your tinnitus make it difficult for you to enjoy life?
13. Does your tinnitus interfere with your job or household responsibilities?
14. Because of your tinnitus do you find that you are often irritable?
15. Because of your tinnitus, is it difficult for you to read?
16. Does your tinnitus make you upset?
17. Do you feel that your tinnitus problem has placed stress on your relationships with members of your family and friends?
18. Do you find it difficult to focus your attention away from your tinnitus and on other things?
19. Do you feel that you have no control over your tinnitus?
20. Because of your tinnitus, do you often feel tired?
21. Because of your tinnitus, do you feel depressed?
22. Does your tinnitus make you feel anxious?
23. Do you feel that you can no longer cope with your tinnitus?
24. Does your tinnitus get worse when you are under stress?
25. Does your tinnitus make you feel insecure?
-
Asia Pacific Journal of Research
ISSN (Print): 2320-5504
ISSN (Online): 2347-4793
www.apjor.com Vol: I. Issue LII, June 2017
17
APPENDIX- II
TINNITUS HANDICAP INVENTORY BANGLA
, , ,
?
?
?
?
?
?
?
, ?
? (, )
?
, ?
?
?
?
?
?
?
?
?
?
-
Asia Pacific Journal of Research
ISSN (Print): 2320-5504
ISSN (Online): 2347-4793
www.apjor.com Vol: I. Issue LII, June 2017
18
?
?
?
?
?
-
Asia Pacific Journal of Research
ISSN (Print): 2320-5504
ISSN (Online): 2347-4793
www.apjor.com Vol: I. Issue LII, June 2017
19
-
Asia Pacific Journal of Research
ISSN (Print): 2320-5504
ISSN (Online): 2347-4793
www.apjor.com Vol: I. Issue LII, June 2017
20
-
Asia Pacific Journal of Research
ISSN (Print): 2320-5504
ISSN (Online): 2347-4793
www.apjor.com Vol: I. Issue LII, June 2017
21
-
Asia Pacific Journal of Research
ISSN (Print): 2320-5504
ISSN (Online): 2347-4793
www.apjor.com Vol: I. Issue LII, June 2017
22
-
Asia Pacific Journal of Research
ISSN (Print): 2320-5504
ISSN (Online): 2347-4793
www.apjor.com Vol: I. Issue LII, June 2017
23